Blog Categories


Posts Tagged ‘azure’

Data and Event Ingest with Cortana Analytics

cortanaanalyticsjustcloudIn my last post, I introduced Microsoft’s Cortana Analytics Suite, a collection
of Azure-based PaaS and SaaS offerings bundled to provide
a comprehensive toolkit for building Advanced Analytics and IoT solutions.  These are cloud-based resources to perform a variety of functions, from collecting telemetry from remote devices, to predicting diabetes-related hospital readmissions, to showing real-time race car metrics to a pit crew.  In most use cases for Big Data or IoT (Internet of Things) solutions, data and event ingest is a critical step, and of course Cortana Analytics has several options to help streamline and support this operation.  This post will provide a summary of those offerings and what functionality they provide.


EventHubFor event ingestion, Azure Event Hubs is a managed event ingestor with broad-based compatibility and elastic scale, capable of logging millions of events per second. Event Hubs sit in the Azure Service Bus, basically at the same level as Topics and Queues, but they provide a different sort of functionality than typical message queuing.

An Azure Event Hub can capture events from Event Publishers (such as incoming device telemetry from wearable devices, continuous feedback from mobile apps, traffic metadata from large scale web farms, etc) using AMQP and HTTP as primary API interfaces.   Downstream applications, as subscribers to the Event Hub, are called Consumer Groups and each have their own view of the event stream, and can process of their own accord — with real-time analytics tools (e.g. Azure Stream Analytics and Power BI) or by storing the data via adapters (e.g. to Azure Blob Storage — more on that later).

Essentially, Azure Event Hubs provide your solution architecture with a scalable and elastic means of decoupling event ingest from event consumption or processing.


AzureDataFactoryThe other means of conducting data workflows is to use Azure Data Factory.  Data Factory is an integration service that allows orchestration and automation of data movement and transformation.  I recently posted about ADF as part of Perficient’s “Azure: Did You Know?” series, so I won’t repeat myself here.  But suffice it to say that Azure Data Factory ends up being somewhat like the circulatory system of the Cortana Analytics Suite, moving data between various storage types and service points.

Azure Data Factory could be used, for example, to build a process workflow that:

  1. takes data from an on-premises SQL Server database
  2. copies it to Azure Blog Storage
  3. then moves it into an HDInsight Hadoop cluster
  4. and trains an Azure ML Model with it

Azure Data Factory can also subscribe to an Azure Event Hub, allowing ADF to process incoming event data directly into a pipeline.

Using either of these services, once event/data ingest has occurred, you’re going to want to store that data.  For this Azure provides a number of solid storage options depending on the needs: Azure Table Storage, Azure Blob Storage, DocumentDB, Azure SQL Database, Azure Data Lake, etc.   Some of these are quite cheap, some are SQL-oriented, some NoSQL oriented.  Additionally, Microsoft has a new Azure service that will help catalog and make discoverable registered data sources.  We’ll talk about what happens AFTER ingest — Storage and Information Management — next time.

AzureCon Recap: IoT Suite Now Available

IoT SuiteIn case you missed it, AzureCon, a one-day virtual conference, was held today. There were 4 keynote speeches from different VP’s at Microsoft and 55 break-out sessions on various Azure topics. A really great idea to do this all virtually and it lived up to the hype! There were a number of key announcements, new services, and updates presented today and I’ll be blogging on the key highlights all this week.

Azure IoT Suite Now Available!

Microsoft first teased the IoT Suite back in May at the Ignite Conference. Today we got our first glimpse at the actual solution. The Azure IoT Suite is a set of pre-configured Azure features and services which spans across PowerBI, HDInsight, Machine Learning, Storm, NoSQL, and many others. These services provide analytics, workflow automation, device connectivity, command and control, and dashboards for all your IoT needs.

L:ast week, I blogged about the IoT Starter Kit and IoT Core Preview. Combined with today’s announcement, Microsoft has now provided the full capability for your end-to-end IoT solution. Read the rest of this post »

Announcing Windows IoT Core Preview and IoT Core Starter Kit

1_iotKitExciting news in the world of IoT! Back in August, Microsoft announced the preview build for the Windows IoT Core operating system for the Raspberry Pi 2 and the MinnowBoard Max.

Windows 10 IoT Core is a new edition for Windows targeted towards small, embedded devices that may or may not have screens. For devices with screens, Windows 10 IoT Core does not have a Windows shell experience; instead you can write a Universal Windows app that is the interface and “personality” for your device. IoT core designed to have a low barrier to entry and make it easy to build professional grade devices. It’s designed to work with a variety of open source languages and works well with Visual Studio.

This is cool stuff. IoT Core supports WiFi and Bluetooth connections, works with Visual Studio 2015, and is compatible with alternative languages such as Node.js and Python. Getting started is simple, and Microsoft even publishes a number of samples on GitHub. Read the rest of this post »

Azure Automation – Announcing Native Powershell Support

Azure automationLast week, Microsoft announced native support for Powershell in Azure Automation. From the Azure blog

If you’re familiar with the Azure Automation service, you know it’s a wonderful tool for reliably automating all the manual, long-running, error-prone, and frequently repeated tasks needed to keep your IT infrastructure in the cloud and on-premises up and running. But you’re probably also aware Azure Automation doesn’t actually execute pure PowerShell, it executes PowerShell Workflow.

There are a lot of benefits to PowerShell Workflow. In addition to allowing you to leverage the semantics, modules, knowledge, content, and community around PowerShell, it also gives you access to the power of Windows Workflow Foundation. With Windows Workflow Foundation you have the ability to checkpoint, suspend, and resume your runbooks to better handle logic, environment, or network failures, as well as the ability to easily execute things in parallel or serially.

Read the rest of this post »

Azure VM – Boot Diagnostics v2

Virtual machineEarlier this month, Microsoft announced support for 2 new debugging features: Console Output and Screenshot support for Azure Virtual Machines v2. When bringing your own image to Azure or even booting one of the platform images, there can be many reasons why a Virtual Machine gets into a non-bootable state. These new features enable you to easily diagnose and recover your Virtual Machines from boot failures.

For more information on how to configure the new features, please visit the Azure blog post here. For assistance implementing your Azure solution, contact us at Perficient and one of our 28 certified Azure consultants can help today!

Azure: Did you know? Application Proxy for Web Applications

Have you ever needed to publish an internal web application for use outside of your corporate firewall?  If you have, you’re probably aware of many of the complications that come up while exploring that option including what ports need to be exposed, to DMZ or not to DMZ, various authentication challenges, etc.  Well, now you can with Azure Application Proxy and the configuration process couldn’t be any easier!

Read the rest of this post »

Azure SQL – Preview Using Azure AD Authentication

Azure SQL DatabaseAzure SQL Database now supports Azure Active Directory authentication in public preview. Azure Active Directory authentication is a mechanism of connecting to Microsoft Azure SQL Database by using identities in Azure Active Directory (Azure AD). With Azure Active Directory authentication you can centrally manage the identities of database users and other Microsoft services in one central location. Central ID management provides a single place to manage SQL Database users and simplifies permission management.

Benefits include: Read the rest of this post »

Azure VM – Announcing the GS Series

Virtual machineEarlier this month, Microsoft announced the GS Series for Azure VM. A variant of G-series, the GS-series, which combines the compute power of G-series with the performance of Premium Storage to create powerful VMs for your most storage and compute intensive applications.

Powered by the Intel Xeon E5 v3 family processors, the GS-series can have up to 64TB of storage, provide 80,000 IOPs (storage I/Os per second) and deliver 2,000 MB/s of storage throughput. The GS-series offers the highest disk throughput, by more than double, of any VM offered by any other hyperscale public cloud provider. Furthermore, both the G-series and GS-series offer up to 20 Gbps of network bandwidth, more than double that of the closest VMs of any hyperscale public cloud provider. Read the rest of this post »

Tags: ,

Posted in Microsoft Azure

Azure Backup – Announcing Backup for Azure IaaS VM’s

Backup, onlineLast week Microsoft announced general availability of Azure IaaS virtual machine backup, previewed earlier this year.

Why use Azure Backup?

  • Azure IaaS VM backup provides application consistent backup for Windows operating systems and file system consistency for Linux operating systems without the need to shut down virtual machines, making it enterprise ready solution.
  • Azure Backup transfers snapshots taken on a VM to a secure, reliable Azure Backup vault and can restore the VM in a single click.
  • Backups taken on virtual machines can be retained for long-term using industry standard GFS based retention policies.

How do I get started? Read the rest of this post »

Why Azure Should Be Part of Your Business Optimization Strategy


Business optimization is all about making investments that will reduce costs while driving efficiency and productivity, and the cloud is a big part of that, because it enables organizations to move quickly and achieve more, while spending less. 42% of IT decision makers planned to increase spending on cloud computing this year, according to Computerworld. On top of that, 57% of Fortune 500 companies already invest in Azure, Microsoft’s cloud computing platform, which offers a growing collection of integrated services.

It’s a question our team hears quite often – why should we move the cloud, or Azure specifically? Customers come to us with a myriad of unique challenges, and of those challenges, a large portion pertain to infrastructure. Perhaps your team is unable to manage a complex application, or you don’t have enough internal resources, or budget required to keep your infrastructure up to date. With on premises, you have both the human cost and the infrastructure cost, typically equating to a larger total cost of ownership than Infrastructure as a Service (IaaS).

Other times, you may want to deliver services that you just can’t do on premises. Size—of your service, budget, or staff—does not limit IT when the platform for custom services is as readily available and broadly deployable as is the web. That’s where Azure Platform as a Service (PaaS) comes into play. PaaS delivers cloud-based application development tools, in addition to services for testing, deploying, collaborating on, hosting, and maintaining applications. You can use Azure PaaS to create multi-tenant applications (services accessed by many users simultaneously), and the open architecture supports integration with legacy applications and interoperability with onsite systems. Read the rest of this post »