Like many companies, Wayfair has leveraged Active Directory (AD) to centrally manage authentication as well as user and computer objects. Schema updates, Organization Unit structure changes, and Group Policy sprawl has led to a vast number of interdependencies and unnecessary complexity. Additionally, a large swath of our infrastructure is tied into AD, which not only adds to this complexity, but also to the critical nature of Active Directory as a whole.

As Wayfair has continued to get larger and larger, limiting the risk of failure on one domain is extremely important. We needed to come up with a way to adhere to our security best practices as well as architect Active Directory in a way to handle continual growth. Out of the many valid approaches to tackling this problem, we narrowed it down to standing up an entirely new, completely greenfield Active Directory forest and domain, dedicated to and focused on employees and workstations. The benefits to this approach are extensive, including implementing more stringent security policies, restructuring OUs to better manage user objects at scale, streamlined DNS zones, and more; all without having to worry about impacting production systems.

While this potential solution had a great number of benefits, it presented a massive challenge: how do we migrate 15,000 users and their computers, seamlessly and without issues, to the new AD domain? Lots of work was being done under the hood, but the real change for the end user would be to change their password to comply with NIST standards. 

Migrating Active Directory Objects Between Domains

Needless to say, an enormous amount of work, planning, and testing went into this project over the past year. All of that time and energy went into building out the new domain, configuring policies, setting up trust, and porting over services from the old domain to the new one. Wayfair’s code is so deeply rooted with the current domain where we needed to do LDAP lookups across our whole codebase to point to the new domain. A year and a half of planning, research, and preparation is the number one reason the migration of users and computers was as seamless as it was. That said, how did we migrate 15,000 user and computer objects in under 5 months?

With Active Directory being a significant part of Microsoft’s software stack, we were certain there was an enterprise-level migration tool provided by Microsoft to facilitate this. Unfortunately, there is only one officially released tool and it is decidedly not enterprise-ready: Active Directory Migration Tool (ADMT).

ADMT is not like the typical Microsoft tool where they provide continual updates and support (Exchange, WSUS, SCCM, Windows Admin Center, for example). Launching and utilizing the ADMT GUI immediately shows how clunky and underdeveloped the tool is. Even more daunting is the complete lack of any PowerShell support. ADMT command line functionality is not object-based in any way; the output is entirely text output to console.

While it will successfully migrate AD objects, the challenge was managing and coordinating the user and computer object migration of 15,000 employees and their individual schedules.

Being that the tool was built and supported with Windows Server 2000-2003, we knew that using solely the GUI would not be sustainable for the whole migration. What else could we do but… automate!

Creating a Wrapper around ADMT via PowerShell

To work through many of the limitations we identified with using ADMT, we first encapsulated the ADMT command within a PowerShell function:

Invoke-Expression "admt.exe User /f:'$UsersToMigrate' /sd:$SourceDomain /td:$TargetDomain /sdc:$SourceDC /tdc:$TargetDC /po:$Password /ps:$SourcePES /to:$TargetOU /dot:$AccountState /fgm:yes /co:merge+movemergedaccounts"

In using the Invoke-Expression cmdlet, we were able to obfuscate the ADMT command, along with building far more user-friendly switches in the PowerShell function with a traditional PowerShell Param() block.

We successfully leveraged this function to migrate all Active Directory objects, including users, computers, and groups, with a few caveats. One limitation of ADMT is that migrating SID history or performing workstation security translations from the command line only happens if it is installed and run directly on a domain controller. This was a non-starter for us due to security concerns. Thus, for SID history and security translations, we were forced to use ADMT’s wizard-driven GUI. This meant nailing down the required coordination and scheduling of employee migrations.

Tracking and Scheduling Migrations

While ADMT has a built-in database that keeps track of all migrated objects, the downside is that it does not track per-employee scheduling data. We created a separate migration and scheduling database that tied into an existing employee database, allowing us to track the migration status on a per-employee basis. Through this approach, we gave employees the ability to self-schedule their migration weeks in advance, at predetermined time slots per day. We yet again leveraged PowerShell to read and write from the database to set scheduling dates as chosen by employees.

This facilitated a clear approach to knowing who to migrate at each time slot, each day.  By wrapping the Invoke-SqlCmd2 cmdlet into another function with a custom SQL query, we could quickly retrieve a list of employees who were scheduled to be migrated each day and at each time slot.

Communication and Information

One of the early challenges with scheduling involved managing confusion around employee expectations and the date or times of their migration. Quite frequently, employees would be surprised that their computer was restarting as part of the ADMT migration. Fortunately for us, our main migration “quarterback” (as we called him), was also in change of our Office 365 Tenant, so he had experience with leveraging email as the main form of communication.

Again with PowerShell, we created a function that utilized built-in .NET classes to generate a customized email with migration information, as well as a calendar invitation to ensure a reminder was in place for the employee:

$msg = New-Object System.Net.Mail.MailMessage

$msg.From = "IT@company.com"
$msg.To.Add("$($user.EmailAddress)")

$msg.Subject = "Domain Migration - $($user.DateTimeStart.DayOfWeek), $($user.DateTimeStart.Month)/$($user.DateTimeStart.Day) at $($user.DateTimeStart | Get-Date -Format 'h:mmtt') EDT"
$msg.IsBodyHtml = $true
$msg.Priority = [System.Net.Mail.MailPriority]::High

$msg.Body = @"
<html><head></head><body>
$($user.DateTimeStart.DayOfWeek), $($user.DateTimeStart.Month)/$($user.DateTimeStart.Day), starting at $($user.DateTimeStart | Get-Date -Format 'h:mmtt') EDT
</body></html>
"@

$SMTPClient = New-Object System.Net.Mail.SmtpClient('SMTP Server','25')
$SMTPClient.Send($msg)

For the invite, we used the StringBuilder class to efficiently build the ICS file:

$str = New-Object System.Text.StringBuilder
$str.AppendLine("BEGIN:VCALENDAR")
$str.AppendLine("VERSION:2.0")
$str.AppendLine("ATTENDEE;CN=`"$($user.DisplayName)`";RSVP=FALSE:mailto:$($user.PrimarySmtpAddress)")
$str.AppendLine("DTEND;TZID=`"Eastern Standard Time`":$($user.DateTimeEnd | Get-Date -Format 'yyyyMMddTHHmmss')")
$str.AppendLine("DTSTART;TZID=`"Eastern Standard Time`":$($user.DateTimeStart | Get-Date -Format 'yyyyMMddTHHmmss')")
$str.AppendLine("UID:$(Get-Date -Format 'yyyyMMddTHHmmss')-$(([char[]]([char]65..[char]90) + 0..9 | sort {Get-Random})[0..7] -join '')@wayfair.com")

With the ICS file created, we could build the email using the same class as above, System.Net.Mail.MailMessage, with a few new parameters:

# Define the email as an invite
$msg.Headers.Add("Content-class","urn:content-classes:calendarmessage")

# Create and set the content type
$contentType = New-Object System.Net.Mime.ContentType("text/calendar")
$contentType.Parameters.Add("method", "REQUEST")
$contentType.Parameters.Add("name", "Meeting.ics")
$calView = [Net.Mail.AlternateView]::CreateAlternateViewFromString($str.ToString(), $contentType)

# Assign the calendar view to the message being sent
$msg.AlternateViews.Add($calView)

We automated the emailing of employees as a way to remind and inform them about their self-chosen date of migration.

Benefits and Closing Thoughts

Even small improvements can have a great impact. For example, with this migration, we were able to impressively simplify our Active Directory spine:

Instead of having 30+ OUs, one for each site, with sub-OUs for categorizing, we flattened the structure. This allowed for reduced Group Policy complexity and for a more consistent user experience across Wayfair’s many offices over Europe and North America.

There are numerous other improvements and benefits; some for the end user and some for the Sysadmins behind the scenes. For Sysadmins, we are on the latest Microsoft forest functional level, have a new and improved GPO footprint, have rebuilt of our core AD services, revamped AD delegation and role-based access, and most importantly, have limited the amount of outages we would have if on one domain. For the end user, faster logins (less GPOs), a password policy where they need to change their password once a year, and a simpler NETBIOS name is available for the domain they are on. Along the way, our biggest concern was to ensure that the migration was as smooth and painless as possible for our end users. This was achieved by rigorous amounts of communication, hand holding, and documentation of the process.

A small but not insignificant factor that helped make this project a success was the  company-wide willingness that empowered us to bring forward new solutions. The importance of Wayfair’s culture, one where different ideas and perspectives are welcomed and listened to, should not be dismissed. Migrating users/computers/servers to a new domain can be a daunting task, but doing so in a way that is streamlined and unobtrusive to the end user is what wound up making this project a success.