Unable to patch Skype on Sonus SBA

I recently was working on a Sonus SBA, patching away, when I found the patch process failing me. The copying, installation, and what appeared to be overall process was generally working but when the last patch (server.msi) was being applied, the process would fail.

Looking at the ASM/server directly I found the Skype server service was not starting – in fact it was unable to fully start as the DynDB database and log files were missing. Odd – must have been dropped during the last patch process, more than likely when the databases were being upgraded. Simple fix (or so I thought) – run Install-CsDatabase -LocalDatabases -Verbose and the missing database and log would be created…or should have been. However, the process failed with an error stating network name not found:

RtcDyn db state is: DbState_DoesNotExist

Dyn Data Path = C:\CsData\RtcDatabaseStore\rtclocal\DynDbPath, Dyn Log Path = C:\CsData\RtcDatabaseStore\rtclocal\DynLogPath.

Creating database rtcdyn from scratch. Data File Path = C:\CsData\RtcDatabaseStore\rtclocal\DynDbPath, Log File Path= C:\CsData\RtcDatabaseStore\rtclocal\DynLogPath.

Clean installing database rtcdyn.

System.IO.IOException: The network name cannot be found.

Now that is just silly – C:\ not found? I could navigate to the path, the permissions matched the FE servers I have in production, so something else was off. Turns out, when the database install process happens local (SBA/SE) or remotely (FE) the installation still uses the \\servername\c$ method to connect and create the databases. In the SBA case, it was hardened by Sonus security template and the C$ was removed.

Blog Image 10.png

It is also interesting to note this impacts tools like SCCM and its ability to push the client – no C$ = no ability to connect. So, in our case, the fix was simple. Add the C$ share to C:\ and re-start the upgrade. To add the share simply right-click in the Shares window, select New Share, and start the share wizard.

Blog Image 11.png

Enter C:\ as the folder path you wish to share.

Blog Image 12.png

Windows will warn you this is a bad idea – acknowledge that you know more than the system by clicking Yes.

Blog Image 13.png

Enter the hidden share name of C$ (and optionally enter a description (the standard being Default Share)).

Blog Image 14.png

Select the second radio button, granting administrators full access and others read. The permissions will be reset after a reboot and selecting the second option allows you to validate/test the process.

Blog Image 15.png

The final result should show the admin share in your list.

As mentioned, this is only a temporary fix – the share is removed when rebooted and the system policies are reapplied. So make sure you perform this workaround just before the patch is installed and all should work as expected.

Blog Image 16.png

RtcDyn db state is: DbState_DoesNotExist

Dyn Data Path = C:\CsData\RtcDatabaseStore\rtclocal\DynDbPath, Dyn Log Path = C:\CsData\RtcDatabaseStore\rtclocal\DynLogPath.

Creating database rtcdyn from scratch. Data File Path = C:\CsData\RtcDatabaseStore\rtclocal\DynDbPath, Log File Path= C:\CsData\RtcDatabaseStore\rtclocal\DynLogPath.

Clean installing database rtcdyn.

Creating database rtcdyn. Attempt: 1

Setting the database rtcdyn to restricted mode.

Database rtcdyn set to mode Restricted.

Setting database options.

Begin transaction.

Creating objects from dbcommon.sql.

Creating database objects.

Executing Types.sql...

Soder on November 28, 2016 asked "Was Sonus notifed, or they simply dontcare at all?" 

Brian then replied "They were not but the issue is with the cmdlet Install-CsDatabase which is part of Skype and not something Sonus created."

Soder replied "Ok, seems I can reply now.So. You wrote
"In the SBA case, it was hardened by Sonus security template and the C$ was removed."
Its Sonus fault. Doing these "hardening" things without doing their homework (plan careefully, do impact analysis, etc. etc.). And as you say the C$ gets removed if the hardening is activated, it means all Sonus equipment owners are affected all over the world. I wouldnt consider this a minor small tiny issue, right?"

Brian replied "Still not in the 'Sonus is at fault' court - the security templates are provided by Microsoft but I will ask for clarity from both Sonus and the Microsoft PG and post a reply. It should be noted that the application of the Security Template - while permanent - is optional."

Soder replied "Well, if the template comes REALLY from Microsoft, thats a different beast (is that like a Security Policy template called Hardened Workstation or something similar?). But still, Sonus allows the enablement / disablement of this template on their GUI, so assume they have tested this before rolling out to customers. I really want to believe they dont just accept everything unconditionally from MSFT and implement without checking its effects first. I know this is a completely optional feature, so it may not affect literally everybody. But as the option is offered to customers without any warning that it may break the whole Sfb update procedure, I am still confident the ball is on Sonus side."

Brian replied "So yes - the security templates are offered from the Microsoft security team and you can apply the XML files to any computer/server - the process stems back to the Windows 2000 days. The key point to remember is the SBA image/process is a managed MS process much like the LPE phones. Sonus and the SBA image are up to the mercy of the MS team and their direction. So, the only question is where did the direction come from for the template - MS or did Sonus do this on their own. That question I have asked and am awaiting the answer."

Mark H on December 1, 2016 stated "I am running Lync 2013 SBA from Sonus and I see the share is disabled as well  but CU are installing properly.
Maybe Microsoft changed something on their installer."

Kevin I replied "Kevin I from Sonus here. The CU Installer has changed with Skype for Business June CU. In previous CU's the updater did not use the SBA C$ share to update. The security template was changed based on customer feedback to remove access to the c$ share. It was changed a couple of years ago with no issues until this CU.

We have implemented a change to our CU installer to restore the share before the CU installs and then remove it afterwards. 

We are also working with Microsoft to understand if there is a way to change the way the CU's are applied to the SBA. Until then the change I mentioned above in the installer will stay to ensure the CU will install moving forward."

Soder replied "Kevin I:
maybe this is not the best platform to discuss this topic, but why there is not a single(!) word about this whole "Security Hardening Template" topic in the 1700+ pages Complete Sonus Documentation PDF ? Regardless of v4.x v5.x or v6.x."

Brian replied "@Soder - have you seen this link? support.sonus.net/.../Applying+an+SBA+Security+Template"

Soder replied "You are right! I didnt find the topic because I was looking for the "hardening" keyword as seen on the button, not searched for generic expression like "template" or "security", as those spit out 1000+ hits on the 1700+ pages document. Yes, I was blind, but organization of content in the Sonus documentation is yet another (saddening) topic.