by February 10th, 2014
NBC and Microsoft recently publicized they are streaming every event of the 2014 Winter Olympics to any iOS, Android, Windows device using Windows Azure Media Services. What is Windows Azure Media Services (WAMS) and how does it work?
WAMS is a cloud optimized edition of the Microsoft Media Platform (MMP) which handles a variety of tasks such as format conversion, media encryption, analytics with on-demand and live streaming capabilities. Microsoft Media Platform is traditionally confined to a server farm but by leveraging Windows Azure WAMS has nearly limitless compute and streaming capabilities.
When considering infrastructure it is important to consider which configuration we will be using. There are two options to consider Infrastructure as a Service and Platform as a Service.
- Infrastructure as a Service (IaaS). Using this method we must setup and configure virtual machines (VM) to connect to our WAMS setup. To utilize IaaS auto-scaling we must create additional VM’s to handle requests when demand is high. This means we must forecast an approximate number of active streaming requests, create the right amount of VM’s to handle the requests and turn on the auto-scale feature to utilize the dormant, yet pre-configured VM’s.
- Platform as a Service (PaaS). Using PaaS there is no extensive VM configuration. After deploying your cloud service and configuring IIS once you can now depend on Azure to auto-scale your cloud service automatically without having to configure additional VM’s for a “just incase” scenario. There will be no need to forecast the number of concurrent requests at any given time. As long as IIS is setup to provide on-demand and live streaming media correctly once then it is setup for your cloud application no matter how great the demand. Essentially by giving up some control in configuration we can save a lot of work. This is the method most likely being utilized to deliver the Olympics.
The setup for live streaming and on-demand will slightly differ in how they are captured and consumed by the public.
- The live streaming setup involves the footage being captured, encoded and then sent to web roles in Azure (typically referred to as an “ingest” server). This can work with a single web role but for redundancy additional web roles can be used. The additional web roles can consume the data as long as they are at a different DNS address. In this situation multiple web roles are probably used for world wide redundancy. As the data is being pushed to the cloud content delivery web roles begin to pull the data and push it to the requesting parties.
- On Demand streaming does not require the high speed capture and encoding of live footage but does require an enormous amount of storage capacity. Every event during the Olympics will be available for on demand streaming, which means every even must be captured and stored in Azure blob storage. Every event is being captured in full HD (1920 x 1080 resolution). You can imagine this will amount to a substantial amount of data, probably several terabytes. While the live streaming web roles need to pull the streaming encoded content, the on demand web roles need to the stream the media files. Sending a full HD stream to a device such as a cell phone with limited bandwidth is not the most efficient distribution process so Azure utilizes a technology called Smooth Stream.
Smooth Stream is a dynamic content delivery technology that will adapt the stream that is sent to the requester based on their bandwidth. It is being utilized for both on-demand and live streaming events. In order to deliver content at a consistent frame rate free of lag or pixelation the video is broken up into small fragments. As the fragments are delivered and played, the time it took to play the fragment as well as the playback quality will be sent back to Azure. If the quality or playback time does not meet standards set on the server then the next fragment will be sent at a lower quality and the process will repeat. If bandwidth increases during the next fragment a higher quality version of the next fragment will be sent. As you can imagine this means every Olympic event needs to be stored in full HD and in several tiers of lower quality fragments to deliver content to every type of device over any kind of bandwidth.
The Olympics is no doubt one of the most watched events of the year. By utilizing dozens of Azure data centers capturing, replicating and delivering content all over the world Microsoft is once again showing the power of what can be accomplished using Windows Azure. Microsoft began streaming the Olympics in 2008 and since has quietly become a media streaming powerhouse with the ability to deliver content to millions at a moments notice.
by January 28th, 2014
Titanfall is a new blockbuster game for the Xbox One. It is being published by Electronic Arts and is due to be released in March 2014. Titanfall is a first person shooter that will have much of its AI hosting , physics calculations, online match making and multi-player dedicated servers hosted in Windows Azure. This means several things:
- Azure’s IaaS provides dedicated servers for multi-player games providing near infinite bandwidth with low server pings and anti-cheat enabled
- Azure’s PaaS is being utilized to provide physics calculations and specialized AI to learn your style of play
- PaaS and dedicated servers auto scale to provide fast dynamic content to players around the world on a consistent scale
Multi-player infrastructure background
Traditionally multi-player games have been played using a client/server paradigm. This paradigm generally involves a computer acting a dedicated server for the game. This dedicated server accepts connections from a specific amount of players and handles communication between the clients/players. The server normally does not perform any game relevant calculations but would act as a central repository where players send update information which would then be distributed and consumed by every client.
Recently the game development community has moved away from the dedicated server model due to operational cost and replaced it with a player-host model. The player-host model essentially means that one player hosts the game and every other player connects to the host. This new paradigm has several disadvantages to network multi-player gaming but was implemented to save costs on running dedicated servers as game hosts. A few of the obvious disadvantages to the player-host model are:
- Inconsistent bandwidth and server lag of the player chosen to be the host
- No anti-cheat enabled on host
- Slower updates / increased lag due to server not being dedicated
- Local player receives faster updates than other players
How Azure fixes this
The dependence on a cloud infrastructure for a fast paced reactionary game is a significant leap of faith. Video games generally run in a continuous loop created by the game engine to repeatedly update all of the game data (AI, particles, physics, player movement, event handling etc.) and then draw that data to the screen. It takes a substantial amount of CPU and GPU power to calculate and render all of the in-game objects at speeds necessary to achieve the target of 60 frames per second.
The developer of Titanfall, Respawn Entertainment, is utilizing Azure PaaS to handle several expensive calculations normally performed by the local host (console or PC). These calculations are typically done on the local host so the player experiences minimal lag. With these calculations off loaded to the cloud and not affecting any game play, it allows the developers to optimize the Xbox One hardware to handle more graphically intense environments. This strategy could also extend the life of the Xbox One even further in the future.
Cloud computing services such as Azure have allowed dedicated servers to once again be economical. With automatic server scaling and incredibly cheap virtual machine costs, the server cost and total hours of man power have been significantly reduced. The more calculations that are performed in the cloud the more you can do with the hardware available. Another way to look at this is, the more calculations you can do in the cloud significantly impacts the entry point for other hardware platforms. If a developer is able to process 90% of intense calculations on an Azure compute cluster then the hardware needed to play the game can be anything from a tablet to a workstation. This has the opportunity to increase the install base substantially.
Games are real time applications that depend on milliseconds and timing. Azure is effectively performing calculations for a real time application and delivering results to multiple parties simultaneously. If the Titanfall launch performs well, expect hundreds of future Xbox One games to utilize Windows Azure in making the cloud (and Azure) a dominant force in multi-player gaming for years to come.
by December 16th, 2013
With the release of Windows Server 2012 R2 comes many great new features, including a improved virtual machine named generation 2.
Generation 2 virtual machines provide quite a few enhancements across the spectrum of Hyper-V VM technology. Perhaps most notable is the removal of legacy emulated hardware. Removal of the legacy network adapter, IDE controller, floppy controller, serial controller (COM ports), and PCI bus, results in a more efficient VM. You should see faster boot times, and quicker installations from .iso. How does a VM boot without these integral components? Where necessary, they have been replaced with software based versions.
Other enhancements include:
- Replaced BIOS with UEFI (Unified Extensible Firmware Interface)
- Faster boot times
- Support for boot volumes up to 64TB (Uses GPT instead of MBR)
- Enhanced Security
- Smaller attack surface
- Secure Boot – Prevents unauthorized firmware, drivers and OS from running during boot.
- Expansion of data and boot disks while VM is running. Nice!
- Complete reliance on VHDX file format resulting in much better performance (VHD’s are no longer supported).
- Enhanced Session Mode
- This allows device redirection and the ability to control display configuration when connected via the Virtual Machine Connection tool.
Some things to keep in mind with generation 2 machines: Read the rest of this post »
by December 5th, 2013
What is Platform as a Service (PaaS)? How does it differ from Infrastructure as a Service (Iaas)?
Let’s start with IaaS. When “The Cloud” first became popular IaaS was the target. The point of IaaS is to migrate a company data center into Windows Azure. This involves converting whatever physical servers you have to Hyper-V virtual machines and upload the contents to Azure(or sending them via FedEx for Microsoft to upload). While this process is cumbersome and time consuming it does work and has some great benefits. The next step in this process is to create a secured link between your former data center and the Azure data center. Doing this will allow your users connectivity to all of their pre-existing applications. In fact, your users should not even notice that the data center has been moved. Everything should operate exactly how it did prior to the migration. Depending on the size of your infrastructure IaaS has the ability to save you time, space, money and will allow IT admins to breathe easy knowing they don’t have to be overly concerned with server hardware failure. From that perspective, IaaS saves you money because you have zero server maintenance / replacement cost. Another benefit of IaaS is that all of your existing third party software is guaranteed to work on IaaS. If a business has an old Citrix application that is critical for the accounting team, then migrating to IaaS will guarantee that application still works as expected. The problem with IaaS is that you are still running an entire infrastructure. IT will still have to manage the servers, Active Directory, patches, and updates. In short, you aren’t benefiting from a lot of the major benefits of the cloud.
PaaS aims to solve that.
PaaS in Azure is synonymous with “Cloud Service”. The target of PaaS is custom business applications that will replace your existing third party software infrastructure. These applications can be written in either .NET or Java. In PaaS you essentially rent the hardware the application runs on. Renting the hardware means you do not have administrative access to the virtual machines powering the application. The permissions are set so that Remote Desktop to the virtual machine is enabled and users can administer IIS.
What is the advantage of renting hardware the application runs on? Why wouldn’t I simply use IaaS to retain tighter control over the environment?
Azure manages the infrastructure powering the application so IT administrators no longer have to worry about it. The infrastructure used to power a global application with millions of users generating massive amounts of traffic (like SnapChat) is massive. The work and time it takes to manage that infrastructure is several times larger. With PaaS you are able to eliminate the management of the environment the app runs on. It is also cheaper to run an application with PaaS than creating dozens upon dozens of virtual machines. Scaling is much easier with PaaS. It is possible to scale to a massive size very quickly(and automatically) with PaaS. To do the same with IaaS, virtual machines must be created in geographically relevant locations.
PaaS provides most of the flexibility that IaaS provides. By sacrificing a little control administrators and developers are able to automate most of the infrastructure maintenance that comes with growing web applications. This not only saves money but allows developers to create new features without having to worry if the infrastructure can handle it. Whether the application is of massive global scale like SnapChat or simply a small application which applies business logic to users; PaaS on Azure maximizes productivity by removing laborious infrastructure maintenance.
by December 3rd, 2013
Xamarin and Microsoft have teamed up to make all other development platforms irrelevant. Xamarin is the creator of popular cross platform development tools that allow developers to create iOS, Android and Windows applications all in C#. With the launch of Visual Studio 2013, Xamarin and Microsoft announced a partnership that will significantly improve the experience of developing, maintaining and updating apps written for any of the major popular platforms (iOS, Android, Windows).
Some of highlights of this partnership include Portable Class Libraries, Visual Studio integration, Azure Mobile Services integration and licensing discounts with free training for all MSDN Subscribers.
Portable Class Libraries (PCL) are libraries of code that can be used in any of your projects. PCL’s have made cross platform development easier than ever before. By using PCL’s you can keep the specific platform code within their respective projects and keep the bulk of your logic within the PCL. Using this method will speed up development, code maintenance and bug fixing considerably.
Previous to the Visual Studio 2013 partnership Xamarin came with its own cross platform development environment, Xamarin Studio. While still very functional it was no Visual Studio. Developers not familiar with Xamarin Studio would still have to take the time to re-learn the tools that were available to them. Now with full Visual Studio integration developers can continue to use the tools they are already comfortable with as well as using the powerful Azure utilities when developing apps that require mobile services.
Windows Azure has become one of Microsoft’s fastest growing platform. It has been experiencing 100% year over year growth and just announced it has been gaining 1,000 new customers per day! Microsoft has built templates specific for Xamarin iOS and Xamarin Android apps so developers can simply download project templates with sample code prepopulated and making API calls to Azure! Creating mobile services has never been easier. For more information on this process, please visit this link.
The final point is one I’m considerably excited about. Along with the Microsoft partnership Xamarin also introduced Xamarin University. For .NET developers that would like to learn more about mobile development Xamarin University is a great place to look. It provides live online classes, tutorials, labs and a certification exam. If you are an MSDN subscriber you have access to Xamarin University for free! A value of over $1400!!! So sign up while there is still space. Class starts January 20th!
by November 26th, 2013
With the newly released Xbox One, Microsoft has been very up front with their quest to take over your living room. One of the main marketing pushes for the Xbox One is that it does more than play games. It integrates with your cable box, it can multi-task by snapping apps to the side (similar to how it is done in Windows 8) and it can run other “apps” while playing games. The pre-installed apps include staples such as Internet Explorer, Sky Drive, Skype, ESPN and Netflix. The full list of can be found here.
There has been a little confusion regarding building apps for the Xbox One. The operating oystem of the Xbox One is a flavor of Windows 8.1, so some within Microsoft claim that any Windows 8.1 app will run on the Xbox One. Other reports claim that support for ANY app might not be true. The reason this has become such a topic of interest is because there is demand!
Is the Xbox One an app platform?
Typically if you develop for a console you are developing a game and require a console dev kit. These are generally expensive but will allow you to test your game on the console before it is sent off to Microsoft to be certified. If any Windows 8.1 app can run on the Xbox One, then there is no need for the dev kits. Or is there? The short answer is yes, there will always be a need for the dev kit. The Xbox One is still an embedded platform. Developers still need to see how their game performs on the actual Xbox One hardware. There is no getting around that.
But what if you aren’t developing a game? What if you are developing an app?
If the Xbox One runs a modified version of Windows 8.1 then you can imagine there is a way to emulate some Xbox One features on an existing Windows 8.1 developer workstation. Better yet, allow developers the ability to create a virtual machine with the Xbox One operating system to test their apps. Kinect for Windows already exists, so integrating Kinect features could be tested as well.
Ever since Apple TV became a “popular hobby” for Apple, developers have been waiting for the chance to create apps on a platform they can access your TV. Prior to the release of the Xbox One, the Apple TV was the closest device for that but the lack of app store has prevented any real innovation. The Xbox One already has an app store and has multi-tasking functionality to make it a viable platform. Microsoft will most likely have to institute a certification process for Xbox Apps so they do not hinder the performance of games but that is a small price to pay for developers to gain access to a users television.
Once the Windows Phone, Windows RT and Windows 8 platforms are consolidated, the Xbox One app store will hopefully be open for developers and help facilitate the next Windows App ecosystem boom.
by November 20th, 2013
“Start-up” and “Microsoft” are usually not said in the same sentence. Microsoft has historically been synonymous with expensive licensing costs which is precisely why most start-ups have gravitated towards Linux. It is free, it is stable and it is powerful. While Windows and Windows Server may never be free, Microsoft has narrowed the gap considerably with Azure.
One of the most important things for start-ups is to get your identity an idea “out there”. Usually this entails a website or web application which will have minor costs associated with it such as purchasing a domain name and finding a company to host your content. Paying for hosted web space can be fairly inexpensive if just starting out, but it still can’t compare with free. To start using Windows Azure will cost nothing. Simply create a Microsoft account and you are given 10 free websites. There is no trial period where it will start to cost money. They are free, forever. The websites can also be any technology, connect to a database, run a blog, forums etc.. The websites are also managed through the Windows Azure management tools which are second to none.
The beauty of Azure is the flexibility it gives you. If a start-up grows and more technology is needed, then build it up in Azure. There are no restrictions, no limits (unless you set them) on what you can build up and tear down at any point in time. If more compute time or storage space is needed then build more virtual machines or create a storage account. If the virtual machines are Windows Server there is no license required, you are simply billed a monthly fee for how long that virtual machine is running. If you choose to also use the Microsoft stack then development tools are provided to you for free with the ability to deploy cloud and mobile services with the touch of a button.
With the low overhead cost Azure provides, Microsoft has never been more friendly to the start-up company and the tools that are provided have never been better (or cheaper)!
by November 5th, 2013
- They are entirely new languages. Developers have to take the time to learn the language and become familiar with the way it works before they can really start to mold themselves with the language
What does this mean for developers? For me it means a few things:
- Finally the dynamically typed parts of the language can be used when preferred and I won’t have to learn an entirely new language
- Typed languages will come with far superior IntelliSense.
On a larger scale this changes my development mindset. In the past I would choose to invest heavily on the .NET aspect of applications. Whether it be an ASP.NET forms application, SharePoint web part, Windows Phone or Windows 8 application; .NET is where I would want to do my development. For web based applications that meant a lot of code behind and on occasion, being tied down by the ASP.NET page life cycle. Suddenly I find myself more interested in writing Windows Phone 8 and Windows 8 applications in HTML/JS/CSS than using C#/XAML.
TypeScript is still very new (still in Preview) and it will not replace the code behind model entirely, but I feel it has the ability to alter the way ASP.NET applications are made, especially if given access to the .NET Framework.
by October 14th, 2013
A Brief Background
In my previous posts I have walked through setting up Hadoop on Windows Azure using HDInsight. Hadoop is an extremely powerful distributed computing platform with the ability to process terabytes of data. Many of the situations when you hear the term “Big Data”, Hadoop is the enabler. One of the complications with “Big Data” is how to purpose it. After all, what is the point of having terabytes worth of data and not being able to use it. One of the most practical uses is to generate recommendations. The amount of data needed to generate good recommendations can not be understated. To process all of that data you need a distributed computing platform (Hadoop) and algorithms to generate the recommendations (Mahout). Mahout is much more than simply a recommendation engine. Apart from recommendations one of my favorite features is frequent itemset mining. Itemset mining is evaluation of sets of item groups that may have high correlation. Have you ever shopped on Amazon? Towards the middle of a product page Amazon will tell you what items are frequently purchased together. That is what Mahout’s itemset mining is capable of.
Installing Mahout on your HDInsight Cluster
There are a couple things we will need to do to install Mahout.
- Enable Remote Desktop on the head node of your cluster
- To enable RDP on your cluster select “Configuration” and at the bottom of your screen select “Enable Remote Desktop”
- Enter in a new username and password for the account. You must also specify a length of time that the RDP user is valid for. The maximum time is 7 days.
- Note: While enabling RDP is a nice feature it does not come without its frustrations. The RDP user you create is not an administrator on the server, it is a standard user. There is also no way to authenticate as an administrator. So you will have to deal with things like IE being in Security Enhanced mode and not being able to use Server Manager. Read the rest of this post »
by October 10th, 2013
Edit: Part 3 using Mahout here
In my previous post I described the basics of HDInsight on Windows Azure and an example of what a Hadoop cluster can do for you.
Without further delay, lets build a cluster! If you don’t already have a Windows Azure account go here and sign up (it’s free!!)
Login to your Azure portal and you will have a dashboard similar to this:
If HDInsight is not initially on the dashboard simply add it by going here and selecting “Try it now” under “Windows Azure HDInsight Preview”. After installing HDInsight should now appear on your dashboard.
To create a cluster select HDInsight; upon clicking “Create an HDInsight Cluster” you will be presented with the following screen. Read the rest of this post »