Microsoft Ignite is next week, and I invite you to stop by booth #2132 to say hello!

As a Skype for Business Launch Partner for new meetings and voice services in Office 365, Perficient’s experts will be available at the booth to discuss our approach and experience transitioning global organizations to Skype for Business. By partnering with Perficient, organizations are eligible to receive funding for Skype for Business pilots and enterprise deployments. Perficient can help ensure success with deployments of Cloud PBX and Cloud PSTN Conferencing services. We’ve developed a solution for organizations that have already been utilizing Office 365 and are looking to further leverage their investment by using the new capabilities of Skype for Business Online. This offer applies to customers who purchase 25-100 eligible Cloud PBX and Cloud PSTN Conferencing Office 365 SKUs, and in return qualify for a fully funded pilot.
By using our delivery methodology, alongside Microsoft’s Skype Operations Framework, our Skype Operations Framework E5 Accelerator offer focuses on the three core areas (Plan, Deliver, and Operate) that are crucial to Cloud PBX or Cloud PSTN Conferencing deployments:

Perficient consistently has been recognized by Microsoft as one of its premier national solution providers, including winning East Region National Solution Provider (NSP) Partner of the Year and Cloud Partner of the Year, Central Region NSP Partner of the Year, and West Region NSP Partner of the Year in 2016. With nationally known experts on Office 365, Azure, and Yammer platforms, as well as a deep and rich history in SharePoint, Skype for Business, and Exchange, Perficient will show attendees how to use Microsoft platforms, products, and best practices to connect employees to key communications and data, and especially to one another.
Stop by booth #2132 to discuss how our team can help with your journey to Skype for Business.
For live updates throughout Microsoft Ignite, follow the Perficient experts online on the Microsoft blog at blogs.perficient.com/Microsoft and on Twitter @Perficient_MSFT.
For more information on our Skype Operations Framework E5 Accelerator offer, please contact sales@perficient.com, and click here to learn more about our Skype for Business practice.

Background

I was recently working on an engagement where the customer was migrating from Exchange 2010 to Office 365 Exchange Online. The customer had been a long time Enterprise Voice customer (OCS, Lync 2010, and now Lync 2013). As the customer grew over the years, so had their Lync environment. The customer originally deployed OCS 2007 R2 for only internal use. The default SIP domain was based off their internal Active Directory namespace (corpnet.domain.com). When Lync 2010 was introduced, the publicly routable SIP domain was added to the Lync topology (domain.com) to clean things up and match their SMTP addresses. Fast forward to 2015, the customer is migrating to Office 365 Exchange Online, which means voicemail and auto attendants were also being migrated to Exchange Online Unified Messaging.
Note, this article will not walk through the actual steps necessary to configure Lync/Skype for Business with Exchange Online Unified Messaging. If you need guidance in this area, refer to this article.

Issue Overview

The issue encountered was internal Lync calls to Exchange Online Unified Messaging (voicemail, auto attendants, etc.) would complete successfully, but any call made from a PSTN caller to Exchange Online Unified Messaging (subscriber access, auto attendants, voicemail) would fail. After reviewing traces of the call, it appeared that the default SIP domain was appended to the From/To fields in the SIP INVITE.   It makes sense why a SIP domain would get appended to these types of calls (so Office 365 knows where to route traffic). The problem is, an administrator cannot control which SIP domain is appended in the INVITE. From my testing, the default SIP domain will always be the domain appended to the PSTNGateway FQDNs when egressing out to Office 365.
From the traces below, you can see that the cause of the failed calls was a result of Office 365 not being able to do a DNS SRV lookup for _sipfederationtls._tcp.corpnet.domain.com as the ms-diagnostics response from Office 365 mentions “Unable to resolve DNS SRV record.”

 

Here is where the problem took a turn for the worse…

1. The customer did not have a public DNS subdomain for corpnet.domain.com. When asked if it was possible to add a public subdomain for corpnet, the admin quickly informed me that they had tried to get the subdomain created in the past, but the public DNS provider could not support the request.
2. Out of curiosity, I attempted to change the PSTNGateway from an FQDN (gateway.corpnet.domain.com) to an IP address (192.168.123.10 in this example).
   • Unfortunately, the default SIP domain was still added to the FROM/TO fields of the SIP INVITE. It was worth a shot, but not surprised by the result.
3. I attempted to create a new PSTN Gateway in the Lync topology with 192.168.123.10@domain.com, but the topology does not allow the SIP domain name to be specified.
   • It was worth a shot, but again, not surprised by the result.

After attempting the three scenarios above, we determined that the only option was to change the default SIP domain.

Changing the Default SIP Domain

Since the public SIP domain was added during the Lync 2010 upgrade, changing the default SIP domain was not as painful as it could have been.  The certificate SAN entries were correct on all the servers (Front Ends, Edges, SBAs, etc.), users already were using the public SIP domain for their SIP addresses, etc..  All we had to do was to execute the set-cssipdomain command to change corpnet.domain.com from the default SIP domain to domain.com.   The figure below is the output of both the get-cssipdomain and set-cssipdomain commands.

Repercussions from Changing the Default SIP Domain

If you read the set-cssipdomain TechNet article carefully, the article mentions “If you change the default SIP domain you will need to restart the RTCCAA and RTCCAS services.”  This statement matches the yellow warnings that appeared after executing the set-cssipdomain command above.  Great, things are moving right along… Oh no, hold on a minute…

Mediation Services

After waiting for the default SIP domain change to replicate, I started to review the Lync event logs on the Front Ends.  There was one warning, one error, and three informational messages that caught my attention.  The informational messages were expected and looked great.  What caught me by surprise was the warning and error message.  From the event logs below, it appeared the Mediation service(s) also had to be restarted as part of the default SIP domain change.

Log Name: Lync Server Source: LS Mediation Server Date: 7/24/2015 6:02:21 PM Event ID: 25029 Task Category: (1030) Level: Information Keywords: Classic User: N/A Computer: GCR-LYNCSTDOBT.corpnet.domain.com Description: Mediation Server applied the settings changes Setting Name: PDP Property Description: ServiceGruu Old Value: sip:gcr-lyncstdobt.corpnet.domain.com@corpnet.domain.com;gruu;opaque=srvr:Microsoft.Rtc.Applications.PDP:cT_X2URjIFW0ygzTE3iGYwAA New Value: sip:gcr-lyncstdobt.corpnet.domain.com@domain.com;gruu;opaque=srvr:Microsoft.Rtc.Applications.PDP:cT_X2URjIFW0ygzTE3iGYwAA

Log Name: Lync Server Source: LS Mediation Server Date: 7/24/2015 6:02:21 PM Event ID: 25029 Task Category: (1030) Level: Information Keywords: Classic User: N/A Computer: GCR-LYNCSTDOBT.corpnet.domain.com Description: Mediation Server applied the settings changes Setting Name: Mediation Server Topology Property Description: DefaultDomain Old Value: corpnet.domain.com New Value: domain.com

Log Name: Lync Server Source: LS Mediation Server Date: 7/24/2015 6:02:21 PM Event ID: 25029 Task Category: (1030) Level: Information Keywords: Classic User: N/A Computer: GCR-LYNCSTDOBT.corpnet.domain.com Description: Mediation Server applied the settings changes Setting Name: MRAS Property Description: ServiceGruu Old Value: sip:gcr-lyncedgpool.corpnet.domain.com@corpnet.domain.com;gruu;opaque=srvr:MRAS:WjJ6-s1PyFeipMlcPV0pwwAA New Value: sip:gcr-lyncedgpool.corpnet.domain.com@domain.com;gruu;opaque=srvr:MRAS:WjJ6-s1PyFeipMlcPV0pwwAA

Exchange UM Auto Attendant Transfers

After the Mediation services were restarted, we proceeded with inbound and outbound PSTN testing.  Calls made from the PSTN to Exchange Online Unified Messaging (subscriber access, auto attendants, voicemail) were now working correctly (yeah!).
We were preparing to migrate the auto attendants to the cloud, but with any Office 365 migration, the existing Exchange environment needs to support the workloads (Unified Messaging, etc.) until they are officially migrated to Office 365.  After further testing, we discovered that the existing on-premises Exchange 2010 auto attendants were unable to complete their transfer requests to Lync.  After reviewing snooper traces, it appeared the on-premises Exchange 2010 UM services processed the transfer request correctly.  The SIP INVITE would get routed to Lync, but Lync would then attempt to route the request out the Lync Edge infrastructure.  As you can see in the screenshots below, the operator transfer request was supposed to be routed to extension 7008 which was a Lync Response Group.  After the default SIP domain change, we started receiving a “404 Not Found.”

Prior to the default SIP domain change, the format for the Exchange contacts were autoattendant.corpnet.domain.com@corpnet.domain.com.  Using the OCSUMUtil utility, we left the Exchange contact FQDNs the same, but decided to update the SIP domain for the Exchange contacts to match the new default SIP domain. The modified format matched autoattendant.corpnet.domain.com@domain.com.  After modifying the SIP domain for the Exchange Contacts, our transfers started to work again.  As you can see in the screenshots below, the same INVITE came into Lync from the on-premises Exchange 2010 UM server, but this time Lync was able to perform a reverse number look up and locate the correct operator Response Group.

Summary

There are many items and tasks to consider when migrating from an on-premises environment to Office 365.  Perficient has performed numerous Office 365 migrations where the default SIP domain was never an issue.  Hopefully you never run into this issue or if you do, you have more flexibility with your public DNS provider.  If you find yourself heading down the same path as this article, remember to:

• Restart the Conferencing Announcement service(s)
• Restart the Conferencing Attendant service(s)
• Restart the Mediation Service service(s)
• Update the SIP domain for all Exchange Contacts to match the new default SIP domain to support the on-premises Exchange Unified Messaging subscriber access and auto attendants during the transition to Office 365

Comments Welcomed.

This is part eleven of a twelve post series, to see an index of all twelve posts click here.
On the eleventh day of Lync’mas my UC Team gave to me:  Eleven Unified Communications resources (web sites, blogs, and social media references).
I could list over 100+ links to my favorite Lync websites and blogs, but following the Lync’mas theme, I’ll try to narrow it down to eleven.  What are we waiting for, lets get started.
1. Lync 2013 Conference

• Microsoft is hosting the first ever Lync conference Feb 19-21st in San Diego, CA.  If you haven’t signed up yet, there is still time.  Perficient is the evening sponsor for the conference, so please find us if you’re planning on attending!

2. NextHop

• A great source for in-depth Lync articles by the premier support engineers and MVPs.

3. Unified Communications Open Interoperability Program

• Microsoft’s voice/video gateway, PBX, and IP-PBX validation matrix with various links to configuration guides.

4. Lync Resource Center

• Central source for everything related to Lync server, client, and Phone Edition updates.

5. Microsoft Lync Certified Support Partners

• A listing of all certified Premier Support Lync Partners (PSLP). 

6. Lync Blog Wiki

• A breakdown of various Lync blogs.

7. Microsoft Lync Pilot Success Kit

• A collection of end user surveys, pre and post pilot presentations, and other templates to use during Lync pilots.

8. Microsoft Lync 2010 Adoption and Training Kit

• Miscellaneous training materials and client-side Lync add-ons.

9. DrRez on Facebook

• Community Facebook channel for Lync related articles and news.

10. DrRez on Twitter

• Community Twitter channel for Lync related articles and news.

11. Microsoft Lync on Twitter

• The official Microsoft Lync twitter handle.

 Merry Lync’mas to all and to all a good night!

This is part four of a twelve post series, to see an index of all twelve posts click here.
On the fourth day of Lync’mas my UC Team gave to me:  Lync Enterprise Voice and Overhead Paging Best Practices.
Before we deep dive into the technical details, you might be wondering how does overhead paging relate to the fourth day of Lync’mas?  When planning Enterprise Voice migrations, we typically work with our customers to integrate Lync certified gateways with their analog devices, including overhead paging systems.  During migrations, it’s important to understand the paging system requirements and how to make the functionality work seamlessly with the best UC solution on the market.   Therefore, the fourth day Lync’mas focuses on FXS/FXO interfaces and loop/ground start signaling… 

Perficient works closely with AudioCodes for Lync Enterprise Voice migrations.  There are numerous gateways available to meet the customer’s requirements.  The table below outlines the various gateways, interface options, and signaling types available for each model.
Best Practice #1: Gather overhead paging system requirements (interface and signaling specs) prior to purchasing Lync certified voice gateways to avoid interface and signaling mismatches between the two platforms.

For the purposes of this blog, I’ll be referencing a Bogen overhead paging system with a Bogen Universal Telephone Paging Interface (UTI1).  The UTI1 is a flexible telephony interface that allows administrators to select the interface that matches the Lync certified gateway interface (pretty sweet huh?!?!).  The picture below displays the numerous options available and the planning that needs to be considered when selecting the appropriate analog gateway.
Best Practice #2: This may not be a “best practice,” but rather a recommendation.  Based on my experience with voice gateways and overhead paging systems, using a FXO interface with loop start typically provides the best end user experience.  Why you ask? I recommend going FXO between the analog gateway and paging system for an “instant answer.”  With FXS, your users will hear 1-2 seconds of ringing and it’s possible that there might be a disconnect delay with the FXS interface which could cause a fast busy sound over the speakers.  A fast busy over the loud speakers for 5-10 seconds is never a good thing. 
What has your experience been with paging systems?  I welcome your comments.

 Best Practice #3:  I’ve seen miscellaneous blogs out there and I’ve even had discussions with technical folks that analog gateways (e.g. AudioCodes MP-11x) can register directly to the Lync Mediation servers for PSTN/analog connectivity.  Does it work and have I seen it work, yes.  It’s important to understand though that there are NO analog voice gateways certified with Lync. Analog voice gateway integration has not been validated by Microsoft and analog voice gateways are not listed on the OIP webpage.  Analog voice gateways must connect through enhanced gateways.  Otherwise, configuration is not supported.  The difference between unsupported vs. supported is distinguished in the architecture examples below:

Best Practice #4: I was going to stop at three best practices, but I present the fourth overhead paging best practice for the fourth day of Lync’mas… Once the overhead paging system has been integrated with the voice gateway, an analog device object should be created in Lync to improve the end user experience.  At the end of the day, it’s all about improving the dialing experience and making the end user happy. 
Below is a PowerShell example demonstrating how to create the overhead paging analog device contact. 

Below is a comparison of the end user experience with and without an overhead paging analog device contact created.  I’ll let you make the call as to which one is easier to remember. 

 Merry Lync’mas to all and to all a good night!

In my original XMPP post, I discussed how to troubleshoot TLS errors relating to Lync and the Microsoft OCS 2007 R2 XMPP gateway. If you believe you’re having TLS related issues, please refer to my previous post. This blog will focus on Lync troubleshooting techniques and Google XMPP responses when establishing federation between the two environments. 

Background

A Lync 2010 architecture was deployed with an XMPP server for instant messaging federation to Google (gmail.com and other Google App domains).  Once the XMPP service was deployed, my customer started receiving the infamous “presence unknown” for Google contacts.  For the purposes of this blog, let’s say the Lync SIP domain (chi.contosouniversity.edu) was used by faculty and the Google Apps domain (gmail.com & contosouniversity.edu) was used by the student population.  Side note… Let’s be real, everyone should be using Lync at this point… Anyways, moving on.
The environment was built to spec, but when Lync users tried to add Google users (gmail.com or contosouniversity.edu) they would simply receive “presence unknown.”  If you didn’t catch it previously, the Lync SIP domain (chi.contosouniversity.edu) is a sub-domain of contosouniversity.edu.  The root domain (contosouniversity.edu) was enabled for Google Apps Talk Service. 
Long story short, the presence issue was caused by a misconfiguration in the Google Apps portal prior to the Lync deployment.  The sub-domain (chi.contosouniversity.edu) was mistakenly enabled for the Google Apps Talk Service.  Unfortunately at the time, I did not have access to the Google Apps Portal.  If you fall into the same boat, below are the troubleshooting techniques that can be used to pinpoint the misconfiguration.
The architecture looks similar to:

 

Troubleshooting Overview

To begin troubleshooting, we’ll first start tracing on the XMPP gateway.

• Log onto the XMPP server
• Navigate to C:\Program Files\Microsoft Office Communications Server 2007 R2\XMPP Gateway\Tracing
• Double-click Snooper

• Once the Office Communications Server 2007 R2 Logging Tool appears, select the following options:
  • OCSXMPPGateway
    • Level – All
    • Flags – All Flags
  • S4
    • Level – All
    • Flags – All Flags
• Finally, click the Start Logging button
• Attempt to send an IM from a Lync user to a Gmail user
  • In the example trace provided below, the Lync user is joe@chi.contosouniversity.edu and the Google Apps user is sarah@contosouniversity.edu

• After the instant message fails, click Stop Logging
• Click View Log Files
  • If you mistakenly click Analyze Log Files, the snooper trace will be blank as shown in the figure below

Once the log file appears, sift through the log and the snippet should appear similar to the one below. 

 
Sifting through the trace, one can quickly tell that the captured trace is not extremely helpful in telling us what exactly caused the communication to fail. While we see a 481-Call Leg/Transaction Does Not Exist and Connection failed to contosouniversity.edu (or gmail.com, etc.) on port 5269 on Connection timer elapsed so removing Outgoing Queue and Outgoing Connection error, one might assume there is a communication issue between the Lync Edge server(s) and XMPP gateway or between the XMPP gateway and Gmail… Well, not so fast.
To ensure firewall rules are open, I first recommend establishing an external telnet session to the external IP address assigned to the XMPP gateway on port 5269. This connection should be successful. Next, I recommend establishing a telnet session from the XMPP gateway to the following Google XMPP servers on port 5269. Again, the majority of these connections should be successful.

• xmpp-server.l.google.com
• alt1.xmpp-server.l.google.com
• alt2.xmpp-server.l.google.com
• alt3.xmpp-server.l.google.com
• alt4.xmpp-server.l.google.com

If all telnet sessions are successful and you’re still questioning why IMs are failing, switch gears and open Network Monitor on the XMPP gateway.  If you were like me previously, you probably do not have the NtSimpleSearch Expert installed.  You can quickly tell by clicking the Experts tab near the top menu.  If all you see is Download Experts, you have your answer.

To make this process easier, navigate to the MSDN NMSimpleSearch website to download the expert.  Once the NMSimpleSearch Expert has been installed, it should appear under the Experts window.

Forget the NMSimpleSearch Expert for a moment and start a new network capture. Send another IM from a Lync user to a Gmail user.  Once the trace is stopped, the cause to our 481-Call Leg/Transaction Does Not Exist error will actually be returned from Google’s XMPP servers.  To find the answer, you can 1) manually read through each packet inspecting the Hex details for the following XML information…
<stream:error><undefined-condition xmlns=”urn:ietf:params:xml:ns:xmpp:-streams”/><str:text xmlns:str=”urn:ietf:params:xml:ns:xmpp-streams”>chi.contosouniversity.com is a Google Apps Domain with Talk Service enabled.</str:text></stream:error></stream:stream>
… or 2) use the NtSimpleSearch Expert to locate error. 
Assuming you want to save yourself the headache, I’ll proceed with option 2. 

• To use NtSimpleSearch, first save the network trace.
  • When saving the file, ensure All captured frames is selected

• Once the trace has been saved, close and reopen Network Monitor
• Open the previously saved network capture
• Select Experts | NtSimpleSearch
• Once the NmSimpleSearch dialog box appears, enter Google Apps in the search field

If we’re troubleshooting the same issue, at least one instance should be found.

Lo and behold, Google is telling the Microsoft XMPP gateway that the domain (chi.contosouniversity.edu) is a Google Apps Domain with Talk Service enabled. 

There you have it!  Simply disable the Talk Service for the appropriate domain(s) in the Google Apps portal and instant messages should start working.

Comments Welcomed.