RPC error when establishing trusts within VMware Workstation Guest domains

A few weeks ago I was setting up an additional forest in my virutal environment so I could test some scripts that needed a functional forest trust. My virtual environment consisted of VMware workstation 5.5.1 and several guest Windows Server 2003 SP1 servers in a single domain.

I followed my usual procedure for prepping Windows Server 2003 SP1 member servers so I installed DNS and verified name resolution between the servers in my primary domain and the workgroup server. Everything worked as expected, so I promoted my workgroup server to a DC in the new forest. Likewise, everything worked as expected and I thought I was well on my way to being able to test my script.

My next step was to establish a forest trust between my primary forest and my new forest. I’ve done this many times before in production environments, so I thought it would be a piece of cake. I proceeded to establish the trust and received an RPC error stating the other domain controller could not be contacted. I thought this was odd, since I could map shares between the DCs and other tests I ran also passed.

Revolutionize Your Business With Generative AI

From product design and software development to virtual agents, content creation, and reporting, GenAI is transforming business. Our AI experts help you unlock GenAI’s full potential and drive growth.

Let’s Get Started

After several frustrating hours of checking security settings, services, and network settings within VMware I resorted to the web to see if anyone else had a similar problem. As it turns out, this appears to be a bug with VMware which is easily worked around. The problem stems from the shared folder driver hgfs.sys, a feature of the VMware tools. If this driver is running during the trust creation, you will likely get the RPC error during trust creation.

The solution is very straightforward. On both DCs that are participating in the trust, navigate to the following registry key:

HKLMSystemCurrentControlSetServiceshgfs

Change the "Start" value to "4" (hex or decimal, its the same). Reboot both DCs, and you should now be able to establish your trusts. If you need the shared folder feature, you can change the start type back to "2" and reboot.

I have not tested trusts with VMware Workstation 5.5.2 or Windows Server 2003 R2, so I don’t know if the problem has been resolved.