Author Archives: m.pomfret

Preparing you server for ShareGate

Introduction

Sharegate can be installed on any computer that has access to the SharePoint site(s). It doesn’t need to be installed on the SharePoint server. The latest version of Sharegate is available for download here.

Installation Steps

  1. Download the installer here, if you are signing up for a trial, you will need to fill the form first here.
  2. Click the Download button to launch the installer.
  3. Read and accept the terms in the License Agreement.
  4. Click install.
  5. Wait a few moments for the installation process to complete.
  6. Sharegate will launch once completed.

Minimum System Requirements

  • Windows Vista, Windows 7, Windows 8, Windows 10, Windows Server 2008/2012,
  • Windows Server 2016
  • Windows Server 2012 R2
  • Microsoft .NET Framework version 4.5
  • 1 GB RAM (minimum)
  • 1 GB hard disk space
  • In order to use Sharegate’s PowerShell module, Windows PowerShell 3.0 or higher is needed.
  • In order to use Sharegate’s browser authentication, Internet Explorer 8 or higher is needed.

Recommended System Requirements

  • 64-bit installation of Windows 7, Windows 8, Windows 10, Windows Server 2008/2012,
  • Windows Server 2016
  • Microsoft .NET Framework version 4.5
  • Quad-core processor
  • 16 GB RAM
  • SSD hard drive with at least 100 GB of free space,
  • In order to use Sharegate’s PowerShell module, Windows PowerShell 3.0 or higher is needed.
  • In order to use Sharegate’s browser authentication, Internet Explorer 11 is recommended.

Server Extension Installation

Generally, the Server Extension is not required to perform your migration, but can be required in some cases. You can get more details about the Server Extension, when it is required, and how to install here.

Activate or Deactivate the License

For instruction on how to activate or deactivate your Sharegate license, see here.

Connect to SharePoint and Office 365

If you are connecting to Central or Global Administration, you need the correct administrative rights.

Index

Establishing a Connection

A Guide to your Connection Options

Connect to Central Administration (SharePoint) (requires administrative rights)

Connect to Global Administration (Office 365) (requires administrative rights)

Connect to your Site Collection

Note: Remember to never connect directly to the web application. For helpful advice on Connection Issues and Alternate Access mappings, please see this blog.

Establishing a Connection

  1. Please take the following steps to connect to your SharePoint environment using Explorer.
  2. Click Add Connection. 

A Guide to Your Connection Options

Site Address:

  1. Insert the Central Administration Address, Global Administrator address, or the Site Collection address.
    • To manage all sites:Use your Office 365 Administration Center or SharePoint Central Administration address.
    • To manage a single site:Use your Site Collection address.

Sign in with:

  • Current Windows User:Connect using your current Windows session credentials.
  • Other User:Connect by manually entering a username and password.
  • Browser:Connect through a Browser Authentication 

Username:

Your username for the Central Administration Address, Global Administrator address, or the Site Collection address.

Password:

Your usual password used to access the Central Administration Address, Global Administrator address, or the Site Collection address.

Use a Proxy

Certain enterprises require the use of proxy servers to connect to your network. When this is the case, you will need to set up a proxy connection in Sharegate. I

How-To

  1. Toggle Proxy
  2. Enter the Host(domain name) of your SharePoint site.
  3. Specify your port number (port80 is used for HTTP traffic and is set by default). For a list of network ports used by key Microsoft server products, see here.
  4. Sign in with:
    1. Default User: Connect using your current Windows session credentials.
    2. Same as SharePoint: Connect using the same credentials as SharePoint.
    3. Other User: Connect by manually entering a username and password.
  5. Enter your Usernameif needed.
  6. Enter your Passwordif needed.
  7. Click Connect
  8. You will then be brought back to General to select your other connection options. For information on setting up a connection, see here.

Connect to Central Administration

Using your Central Administration port will allow you to have access to your entire SharePoint on-premises Farm. Use Central Administration to manage all site collections. 

  1. Enter the name of your server followed by your port number.
  2. Enter your password in the Password
  3. Click Connect
  4. Depending on the address you used, you may be prompted to connect to all of your Site Collections. This is normal, it occurs because we frequently copy the address from our address bar, and that address is associated to a specific site in the Central Administration, but not the Central Administration itself.
  5. Click on Manage all Sites(If you used the correct address, you will not see this message)
  6. You will now be connected to your Farm throughCentral Administration

Accessing your port number

To complete this action, you must have administrative rights.

When you first build your SharePoint farm, the Configuration Wizard automatically generates a five digit port number. Some users choose to modify it through the SharePoint Management Shell.

To access this number, using the server running SharePoint:

  1. Click Start
  2. Go to Administrative Tools
  3. Click SharePoint Administration
  4. Connect to Central Administration.
  5. Enter your address in the Site Addressfield

Connect to Global Administration (Office 365)

Using your Global Administrator Name will give you access to your SharePoint Administration Center on Office 365. Use your Global Administrator to manage all site collections.  

  1. Enter your Global Administration address. The format is usually “company-admin.sharepoint.com”.
  2. Enter your password in the Password
  3. Click Connect
  4. You will now be connected to your Global Admin Center.

Access your Global Administration Name

To complete this action, you must have administrative rights.

The Global Administration Name can be accessed through the Azure AD portal or by using PowerShell.

For a complete guide on accessing your Global Administration Name, please click here.

A quick backdoor way to access the Global Administration Name is to sign into your Office 365 and find it in your address bar.

This will tell you:

  • Global Administration name:gsoft (gives you access to entire Office 365 Global Administration)
  • Global Administation root address:sharepoint.com (gives you access to entire Office 365 Global Administration)
  • SharePoint admin center address:gsoft-admin.sharepoint.com  (gives you access to entire Office 365 Tenant and is used to manage Sharepoint Online Administration Center)

Connect to your Site Collection

When you connect to your site collection, you are not accessing your entire Farm or Tenant. This connection method allows you to manage only one site collection. 

  1. Enter your site address in the Site Address 
  2. Enter your password in thePassword 
  3. Click Connect
  4. Since you can only access all Site Collections through connecting to your Central Administration, you will be prompted to Continue, or Change Connection.
  5. Click Continue.
  6. You will now be connected to your Site Collection.

Connect with Browser (Browser Authentication)

Connecting to your environment with Browser is useful in the following situations:

  • You are encountering network issues.
  • The other authentication methods are not working.
  • The site address of the environment is not linked to your windows account (when this scenario occurs with an Office 365 environment, Sharegate will set the Authentication to Browser by default).

Please Note:

If you are connecting to Central or Global Administration, you need the correct administrative rights.

  1. Once you have accessed the connection screen, type your Site Address.
  2. Select Browseras your Authentication
  3. Click Next.
  4. Sharegate will automatically open a browser authentication window.
  5. Type the email address associated to the account.
  6. Click Next.
  7. Type the password associated to the account.
  8. Click Sign in.
  9. You can opt to stay signed in or not by clicking Noor Yes.
  10. You will now automatically be redirected back to Sharegate where you can access the environment.

Use a Proxy

Certain enterprises require the use of proxy servers to connect to your network. When this is the case, you will need to set up a proxy connection in Sharegate. If you would like to know more on proxy servers, see here.

Prerequisite(s)

You have accessed your Connect screen.

How-To

  1. Toggle Proxy
  2. Enter the Host(domain name) of your SharePoint site.
  3. Specify your port number (port80 is used for HTTP traffic and is set by default). For a list of network ports used by key Microsoft server products, see here.
  4. Sign in with:
    1. Default User: Connect using your current Windows session credentials.
    2. Same as SharePoint: Connect using the same credentials as SharePoint.
    3. Other User: Connect by manually entering a username and password.
  5. Enter your Usernameif needed.
  6. Enter your Passwordif needed.
  7. Click Connect
  8. You will then be brought back to General to select your other connection options. For information on setting up a connection, see here.

General

Performance

The performance setting allows you to choose how many requests can be sent simultaneously to your SharePoint server.

Use the radio buttons to select your preferred option.

Before you select a performance setting, consider the following:

  • It’s recommended to test the different levels to determine which one is the most appropriate for your SharePoint environment (for example, choosing High may slow down your SharePoint server)
  • The Performance setting can be changed during a working process (Reports, actions, edit, or find) and the changes will take effect immediately.

Permissions

For full functionality, we recommend Site Collection Administrator rights (even if you are a Farm Administrator).

To allow Shargate to auto-assign Site Collection Administrator, toggle on (toggle off is the default).

This allows Sharegate to automatically grant your user account the required rights when you access a site collection.

Note: This option only works when you are connected directly to your SharePoint farm. This applies to the user account that is used to connect to the farm as Site Collection Administrator. Permissions added in this way will not be removed automatically. For information on how to undo this action once applied, click here.

Diagnostic Mode

Diagnostic Mode allows you to reproduce an error and send a diagnostic report to Support. This gives them the capability to see exactly when, where, and why something is going wrong.

Appendix

Ports & Firewall

Which ports does Sharegate require to be open in our firewall?

Sharegate requires two basic ports:

  • HTTP (80)
  • HTTPS (443)

If you are migrating to a SharePoint Online or Office 365 destination with Insane Mode, whitelist the following URLs:

For Sharegate licensing, updates, and telemetry, whitelist the following URL:

  • *.share-gate.com/

 

Read More

Office 365 Import service – Use network upload to import SharePoint data to Office 365

Use the Office 365 Import service and network upload to import files and documents to SharePoint Online and OneDrive for Business sites in your Office 365 organization.

When you use network upload, you copy a SharePoint Online migration package (which consists of data and manifest files) to an Azure storage location in the Microsoft cloud. Then the Office 365 Import service copies the files from the storage area to the target site in Office 365.

Using network upload is the most efficient way to import documents to SharePoint Online team sites.

Here are the steps for using network upload to import SharePoint files to sites in your Office 365 organization:

Step 1: Install the SharePoint Online Management Shell and the Azure AzCopy tool

Step 2a: Setup Working Directories

Step 3: Determine your locations and credentials

Step 4a: Export the Web / Site Collection using the below cmdlet

Step 4b: Create a SharePoint Online migration package

Step 5: Prepare the SharePoint Online migration package

Step 6: Upload the migration package to Office 365

Step 7: (Optional) View a list of the data and manifest files uploaded to Office 365

Step 8: Create a data import mapping file

Step 9: Create a data import job in Office 365

Before you begin

You have to be a global administrator in your Office 365 organization to access the Import page.

  • Valid Office 365 subscription
  • Valid Microsoft Azure subscription
  • Install SharePoint Online management shell version 16.0.4017.1200 or above fromhere. More details at https://technet.microsoft.com/en-us/library/fp161372.aspx
  • Run SharePoint Online management shell as Administrator and connect SPO service using the below cmdlet

Connect-SPOService -Url https://<TENANT>-admin.sharepoint.com

  • Replace<TENANT> with actual tenant prefix, <SPOSite> with SPO Site prefix and <On-PremSPSite> with SP On-Prem Site Prefix in all PowerShell cmdlets.

Step 1: Install the SharePoint Online Management Shell and the Azure AzCopy tool

The first step is to install the tools you’ll use to upload your files to Office 365. You’ll use the SharePoint Online Management Shell in Steps 3 and 4 to create and prepare a migration package, and then you’ll use the Azure AzCopy tool to upload them to a temporary Azure storage location in the Microsoft cloud. You only have to do this step once. After you install these tools, follow Step 2 through Step 8 each time you want to import a batch of SharePoint or OneDrive for Business files.

IMPORTANT: You have to use Azure AzCopy version 5.0 to successfully upload SharePoint data to the Azure storage location. Newer versions of the AzCopy tool aren’t supported for importing data to Office 365. Be sure to download the AzCopy tool from the Upload files over the network page by following the procedures in this step.

  • Go to https://protection.office.com.
  • Sign in to Office 365 using the credentials for an administrator account in your Office 365 organization.
  • In the left pane, click Data governanceand then click Import.
  • On the Import page, click Go to the Import service.
  • On the Import data to Office 365 page, click New job , and then click Upload SharePoint or OneDrive data.
  • On the Upload files over the network page, in step 3, click Download the SharePoint Online Management Shell.
  • Download and install the SharePoint Online Management Shell.
  • On the Upload files over the networkpage, in step 4, click Download tool to download and install the Azure AzCopy tool.
  • In the pop-up window, click Runto install AzCopy.
  • Keep the Upload files over the networkpage open. You’ll use it in the next step to create URLs and containers names.

Step 2a: Setup Working Directories

You need the following directories on your local machine:

$sourcePackage : a shared directory which contains content files

$sourcePackagePath: a local directory which contains XML files

$targetPackage: a local directory which contains ready to upload package

For example, here are my values for these parameters:

$sourcePackage = ‘\<Server01>tempexportedLibrary.cmp’  #Content file

$sourcePackagePath = “C:spmigrationsourceexport.cmp” #XML files # contains upload package

$targetPackage = ‘C:migrationtarget’ # contains XML files

  • <Server01> is the name of the server which I have shared the folder, and because I want to do a full export without any compression, I specified the directories for content files and xml files as export.cmp

Step 3: Determine your locations and credentials

In this step you need to identify your locations and credentials, including the location of your source files, target files and web location.

On your local computer, open the SharePoint Online Management Shell. Run the following commands substituting your values.

Step 3: Determine your locations and credentials

Step 4a: Export the Web / Site Collection using the below cmdlet

In this step, we will produce the content and xml meta data out of the SharePoint document library.

Export-SPWeb -Identity $sourceWeb -path $sourcePackagePath -ItemUrl $sourceDocLib –NoFileCompression

  • RDP to a WFE for On Premise SharePoint Server
  • Open SharePoint Management Shell as administrator
  • Export the Web / Site Collection using the below cmdlet

Export-SPWeb -Identity “http://<On-PremSPSite>” -ItemUrl “/OnPremDocLib” -Path “C:SPOnPremExport” -NoFileCompression -Inc

We use Export-SPWeb to export the contents of the document library along with XML meta data.

Here is the description of the parameters:

  • $sourceWeb : the on premise SharePoint site URL
  • $sourcePackagePath : where you want to store the files
  • $sourceDocLib : the relative path to the document library that you want to export , like ‘/mydocs/’
  • Youhave to use NoFileCompression switch

When you run the above command to export a document library from on premise SharePoint, it will create a directory in $sourcePackagePath with both data and xml files.

You have to browse to that location;

  • Copy all the content files (*.dat) to $sourcePackage folder and
  • Copy all the content files (*.xml) to $sourcePackagePath.

Step 4b: Create a SharePoint Online migration package

The next step is to create a migration package that’s based on the files that you want to upload to the Azure storage location. The migration package also contains metadata files that contains the information used to upload the data files. This procedure assumes the files that you want to import to Office 365 are located on a file share or file server in your organization.

  • On your local computer, open the SharePoint Online Management Shell from the Startpage, and then run the following command:

Run the following command:

$credentials = Get-Credential

  • In the Windows PowerShell Credential Requestdialog box, type the user name and password for your Office 365 administrator account, and then click OK.

Run the following command to connect the shell to your SharePoint Online organization:

Connect-SPOService -Url https://<Your organization name>-admin.sharepoint.com -Credential $credentials

  • In the SharePoint Online Management Shell, run the following command to create a migration package.

New-SPOMigrationPackage -SourceFilesPath <File share or server where data files are located> -OutputPackagePath <Location on file share to copy the package to> -TargetWebUrl <Target SharePoint or OneDrive for Business site> -IgnoreHidden -ReplaceInvalidCharacters

The following table describes the parameters used in this command, and their required values.

Parameter Description Example
SourceFilesPath The path for the file share or file server that contains the data files that will be uploaded to Office 365. -SourceFilesPath “c:Share1DataFilesDataImportJob1”

Or

-SourceFilesPath “\FILESERVER01SPFilesDataImportJob1”

OutputPackagePath The location where the metadata files for the migration package will be copied to. -OutputPackagePath “c:Share1DataFilesImportJob1Package”
TargetWebUrl The URL for the Web site in your SharePoint Online organization that you’re importing the data files to. This could be the URL for a SharePoint or OneDrive for Business site. -TargetWebUrl https://contoso.sharepoint.com/sites/SPOCorpSite
IgnoreHidden Use this optional switch to ignore hidden files and folders. Using this switch might help reduce the size of a migration package. -IgnoreHidden
ReplaceInvalidCharacters Use this optional switch to replace characters in file and folder names that would be invalid in SharePoint and OneDrive for Business. For a list of these characters, see Invalid characters in file or folder names or invalid file types in OneDrive for Business. -ReplaceInvalidCharacters

Here’s an example of the syntax for the New-SPOMigrationPackage cmdlet using actual values for each parameter:

New-SPOMigrationPackage -SourceFilesPath -SourceFilesPath “\FILESERVER01FilesDataImportJob1” -OutputPackagePath “c:Share1DataFilesImportJob1Package” -TargetWebUrl https://contoso.sharepoint.com/sites/SPOCorpSite -IgnoreHidden -ReplaceInvalidCharacters

Leave the SharePoint Online Management Shell open. You’ll use it in Step 4 to prepare the migration package so you can upload it to Office 365.


Step 5: Prepare the SharePoint Online migration package

After you create the migration package (from source files on file share, a file server, or an on-premises SharePoint site), the next step is to prepare the migration package by running the ConvertTo-SPOMigrationTargetedPackage cmdlet in your SharePoint Online organization. This cmdlet verifies that the migration package and the source files match, and then validates the SharePoint Online site and document library that you want to import the data files to. The cmdlet creates a target package that will be uploaded to the Azure storage location in Step 5.

  • In the SharePoint Online Management Shell, run the following command. See the previous stepfor instructions on how to connect to your SharePoint Online organization.

ConvertTo-SPOMigrationTargetedPackage -SourceFilesPath <Location of source files> -SourcePackagePath <Location of  metadata files> -OutputPackagePath <location to copy the target package to> -TargetWebUrl <Target SharePoint or OneDrive for Business site> -TargetDocumentLibraryPath <Target document library> -Credential $credentials

The following table describes the parameters used in this command, and their required values.

Parameter Description Example
SourceFilesPath The path for the file share or file server that contains the data files that will be uploaded to Office 365. Use the same value that you used for the SourcesFilesPath parameter when you ran the New-SPOMigrationPackage in Step 3. -SourceFilesPath “\FILESERVER01SPFilesDataImportJob1”

Or

-SourceFilesPath “c:Share1DataFilesContosoCorpSite”

SourcePackagePath The location where the metadata files for the migration package will be copied to. -SourcePackagePath “c:Share1DataFilesContosoCorpSite”
OutputPackagePath The location on the file share or file server where the metadata files for the target migration package produced by this cmdlet will be copied to. -OutputPackagePath “c:SPFilesDataImportJob1TargetPackage”
TargetWebUrl The URL for the Web site in your SharePoint Online organization that you’re importing the data files to. Use the same value that you used for the TargetWebUrlparameter when you ran the New-SPOMigrationPackage in Step 3. -TargetWebUrl https://contoso.sharepoint.com/sites/SPOCorpSite
TargetDocumentLibraryPath The name of the document library (located in the site specified by the TargetWebUrl) that the data files will be imported to. -TargetDocumentLibraryPath “Shared Documents”

Here’s an example of the syntax for the ConvertTo-SPOMigrationTargetedPackage cmdlet using actual values for each parameter:

ConvertTo-SPOMigrationTargetedPackage -SourceFilesPath “\FILESERVER01SPFilesDataImportJob1” -SourcePackagePath “c:Share1DataFilesImportJob1Package” -OutputPackagePath “c:SPFilesDataImportJob1TargetPackage” -TargetWebUrl https://contoso.sharepoint.com/sites/SPOCorpSite -TargetDocumentLibraryPath “Shared Documents” -Credential $credentials


Step 6: Upload the migration package to Office 365

Now you’re ready to use the AzCopy.exe tool (installed in Step 1) to upload the migration package (that you created and prepared in Step 4 and Step 5) to Office 365. This tool uploads and stores the data and manifest files to the Azure storage location for your organization. You’ll run the AzCopy tool twice: once to upload the data files to the files container and once to upload the manifest files to the manifest container.

  • Open a Command Prompt (Run as administrator) on your local computer.
  • Go to the directory where you installed the AzCopy.exe tool in Step 1. If you installed the tool in the default location, go to %ProgramFiles(x86)%Microsoft SDKsAzureAzCopy.
  • Run the following command to upload the data files to the file container in the Azure storage location.

AzCopy.exe /Source:<SourceFilesPath> /Dest:<File container SAS URL> /V

The following table describes the parameters and their required values. Note that the information you got in the previous step is used as the values for these parameters.

Parameter Description Example
/Source: Specifies the path for the file share or file server that contains the data files that will be uploaded to Office 365. Use same value that you used for the SourcesFilesPath parameter when you ran the New-SPOMigrationPackage in Step 4. /Source:”\FILESERVER01SPFilesDataImportJob1″

Or

/Source:”c:Share1DataFilesContosoCorpSite”

/Dest: Specifies the SAS URL for the file container that you got in Step 2. Be sure to surround the value of this parameter with double-quotation marks (” “). /Dest:”https://4lalaf4068be4f67b2c8f44.blob.core.windows.net/files-2016062818-1ab5ecbab35d436d83d2694db206daa5?sv=2012-02-12&se=2016-09-26T18%3A32%3A10Z&sr=c&sp=wl&sig=XQ6gG7%2B20n%2FRhVEvxwzmKrocks2pGNr8J%2FhDLqL2xbU%3D”
/V Outputs verbose status messages into a log file. By default, the verbose log file is named AzCopyVerbose.log in %LocalAppData%MicrosoftAzureAzCopy (for example, C:UsersadminAppDataLocalMicrosoftAzureAzCopyAzCopyVerbose.log). /V
/S This optional switch specifies the recursive mode so that the AzCopy tool will copy files that are located in subfolders in the source directory that is specified by the /Source: parameter. /S

Here’s an example of the syntax for the AzCopy.exe tool using actual values for each parameter:

AzCopy.exe /Source:\FILESERVER01SPFilesDataImportJob1 /Dest:”https://4lalaf4068be4f67b2c8f44.blob.core.windows.net/files-2016062818-1ab5ecbab35d436d83d2694db206daa5?sv=2012-02-12&se=2016-09-26T18%3A32%3A10Z&sr=c&sp=wl&sig=XQ6gG7%2B20n%2FRhVEvxwzmKrocks2pGNr8J%2FhDLqL2xbU%3D” /V:C:UsersAdminDesktopAzCopy1.log

After you run the command, status messages are displayed that show the progress of uploading the data files. A final status message shows the total number of files that were successfully uploaded.

Run the following command to upload the target package to the manifest container in the Azure storage location..

AzCopy.exe /Source:<OutputPackagePath> /Dest:<Manifest container SAS URL> /V

The following table describes the parameters and their required values. Note that the information you got in the previous step is used as the values for these parameters.

Parameter Description Example
/Source: Specifies the path for the file share or file server that contains the target package that you created in Step 4. Use same value that you used for the OutputPackagePath parameter when you ran the ConvertTo-SPOMigrationTargetedPackage. /Source:”c:SPFilesDataImportJob1TargetPackage”

Or

/Source:”c:Share1DataFilesContosoCorpSite”

/Dest: Specifies the SAS URL for the manifest container that you got in Step 2. Be sure to surround the value of this parameter with double-quotation marks (” “). /Dest:”https://4lalaf4068be4f67b2c8f44.blob.core.windows.net/manifest-2016062818-1ab5ecbab35d436d83d2694db206daa5?sv=2012-02-12&se=2016-09-26T18%3A32%3A11Z&sr=c&sp=wl&sig=1%2BGHYQ5nuwTonPxyzxyzR4d9dyxHicEFR2KWHORLzFU%3D”
/V Outputs verbose status messages into a log file. By default, the verbose log file is named AzCopyVerbose.log in %LocalAppData%MicrosoftAzureAzCopy. Each time you run AzCopy.exe and specify this parameter, the verbose status message will be appended to the log file. /V
/S This optional switch specifies the recursive mode so that the AzCopy tool will copy files that are located in subfolders in the source directory that is specified by the /Source: parameter. /S

Here’s an example of the syntax for the AzCopy.exe tool using actual values for each parameter:

AzCopy.exe /Source:”c:SPFilesDataImportJob1TargetPackage” /Dest:”https://4lalaf4068be4f67b2c8f44.blob.core.windows.net/manifest-2016062818-1ab5ecbab35d436d83d2694db206daa5?sv=2012-02-12&se=2016-09-26T18%3A32%3A11Z&sr=c&sp=wl&sig=1%2BGHYQ5nuwTonPxyzxyzR4d9dyxHicEFR2KWHORLzFU%3D” /V:C:UsersAdminDesktopAzCopy1.log


(Optional) Step 7: View a list of the data and manifest files uploaded to Office 365

As an optional step, you can install and use the Microsoft Azure Storage Explorer (which is a free, open source tool) to view the list of the data files and the manifest files that you’ve uploaded to the Azure storage location in Step 5. A good reason to do this is to verify that source files and the manifest files from your organization were successfully uploaded to the Azure storage location.

NOTE: The Microsoft Azure Storage Explorer is in Preview.

To install the Azure Storage Explorer and connect to your Azure storage area:

Download and install the Microsoft Azure Storage Explorer tool.

Start the Microsoft Azure Storage Explorer.

Right-click Storage Accounts in the left pane, and then click Connect to Azure storage.

In the box under Connect to Azure storage, do one of the following based on which container you want to view.

Paste the file container SAS URL (from Step 2) to view the file container.

Paste the manifest container SAS URL (from Step 2) to view the manifest container.

Click Next.

On the Connection summary page, you can review the connection information, and then click Connect.

NOTE: You’re assigned the Write and List permissions. This means you can upload files to the Azure storage location, and then view a list of those files. You aren’t assigned permissions to delete files or containers.

Under Storage Accounts, expand the (Service SAS) node, and then expand the Blob Containers node.

Right-click the files container or the manifest container (depending on which one you connected to), and then click Open Blob Container Editor.

The Azure storage area, with a list of the source or manifest files that you uploaded in Step 5 is displayed.

Repeat steps 3 – 8 to connect to the other container.

When you’re finished using the Microsoft Azure Storage Explorer, right-click the files or manifest container, and then click Detach to disconnect the container from your Azure storage area. Otherwise, you’ll receive an error the next time you try to attach to that container.

Step 8: Create a data import mapping file

After the data and manifest files have been uploaded to the Azure storage location for your Office 365 organization, the next step is to create a comma separated value (CSV) file that specifies the SharePoint or OneDrive for Business site that the data files will be imported to. This file also specifies of the location of the data and manifest files that you uploaded to the Azure storage location in the previous step. You’ll submit the data mapping file the next step when you create the import job.

Download a copy of the CSV Import mapping file.

Open or save the CSV file to your local computer. The following example shows a completed import mapping file (opened in NotePad). It’s much easier to use Microsoft Excel to edit the CSV file.

Workload,FilePath,Name,Mailbox,IsArchive,TargetRootFolder,SPFileContainer,SPManifestContainer,SPSiteUrl

SharePoint,,,,,,files-2016062818-1ab5ecbab35d436d83d2694db206daa5,manifest-2016062818-1ab5ecbab35d436d83d2694db206daa5,https://contoso.sharepoint.com/sites/SPOCorpSite

The first row, or header row, of the CSV file lists the parameters that will be used by the Import service to import files to SharePoint or OneDrive for Business sites . Each parameter name is separated by a comma. Each row under the header row represents the parameter values for importing files. Be sure to replace (or remove) the placeholder data in the mapping file with your actual data.

IMPORTANT: The mapping file can only contain one data row under the header row. This row will contains the information about the migration package that you uploaded to the Azure storage location. Be sure to create a separate mapping file for each migration package you created and uploaded to the Azure storage location. You can submit multiple mapping files when you create an import job in Step 8.

Use the information in the following table to populate the CSV file with the required information.

Parameter Description Example
Workload Specifies the Office 365 service that data will be imported to. To import data files to SharePoint Online or OneDrive for Business, use SharePoint. SharePoint
FilePath For importing files to SharePoint or OneDrive for Business, leave this parameter blank. Not applicable
Name For importing files to SharePoint or OneDrive for Business, leave this parameter blank. Not applicable
Mailbox For importing files to SharePoint or OneDrive for Business, leave this parameter blank. Not applicable
IsArchive For importing files to SharePoint or OneDrive for Business, leave this parameter blank. Not applicable
TargetRootFolder For importing files to SharePoint or OneDrive for Business, leave this parameter blank. Not applicable
SPFileContainer Specifies the file container name that you got in Step 2. files-2016062818-1ab5ecbab35d436d83d2694db206daa5
SPManifestContainer Specifies the manifest container name that you got in Step 2. manifest-2016062818-1ab5ecbab35d436d83d2694db206daa5
SPSiteUrl Specifies the SharePoint or OneDrive for Business site that the files will be imported to. Use same value that you used for the TargetWebUrl parameter when you ran the ConvertTo-SPOMigrationTargetedPackage in Step 4. https://contoso.sharepoint.com/sites/SPOCorpSite

NOTE: Don’t change anything in the header row, including the Exchange parameters; they will be ignored during the SharePoint data import process.

Step 9: Create a data import job in Office 365

The last step is to create the data import job in the Import service in Office 365. As previously explained, you can submit multiple data mapping files (that you created in the previous step) for a single import job. After you create the new import job, the Import service will use the information in the mapping files to import data (that you uploaded to Office 365 in Step 5) to the specified SharePoint or OneDrive for Business site.

Go to https://protection.office.com.

Sign in to Office 365 using the credentials for an administrator account in your Office 365 organization.

In the left pane, click Data governance and then click Import.

On the Import page, click Go to the Import service.

On the Import data to Office 365 page, click New job  , and then click Upload SharePoint or OneDrive data.

On the Upload files over the network page, click the I’m done uploading my files and I have access to the mapping file check boxes, and then click Next.

Type a name for the data import job, and then click Next.

Click Add  to select an import mapping file that you created in Step 7.

After the name of the CSV file appears in the list, select it and then click Validate to check your CSV file for errors.

The CSV file has to be successfully validated to create a data import job. If the validation fails, click the Invalid link in the Status column. A copy of your mapping file is opened, with a error message for each row in the file that failed.

Add and validate additional mapping files, if applicable.

When the mapping files are added and validated, read the terms and conditions document, and then click the checkbox.

Click Finish to submit the job.

The job is displayed in the list of import jobs on the Import data to Office 365 page.

Select the job and click Refresh  to update the status information that’s displayed in the details pane.

In the details pane, click View details to get the latest status for the selected data import job.

 

Read More

FAQ about importing SharePoint data to Office 365

Applies To: Office 365 Admin

This article is for administrators. Are you trying to upload your files to SharePoint Online or OneDrive for Business? See Upload files to a library

Here are some frequently asked questions about using the Office 365 Import Service to import SharePoint data to SharePoint Online and OneDrive for Business sites in your Office 365 organization. For more information importing SharePoint data, see Overview of importing PST files and SharePoint data to Office 365.

Using network upload to import SharePoint data

For step-by-step instructions, see Use network upload to import SharePoint data to Office 365.

Where is network upload available?

Network upload is currently available in the United States, Canada, Brazil, the United Kingdom, Europe, India, East Asia, Southeast Asia, Japan, and Australia. Network upload will be available in more regions soon.

What is the pricing for importing SharePoint files by using network upload?

Using network upload to import SharePoint data is free.

How long does it take to import data to a SharePoint Online or OneDrive for Business site?

It depends on the capacity of your network, but it typically takes several hours for each terabyte (TB) of data to be uploaded to the Azure storage area for your organization. After the data files are copied to the Azure storage area, files are imported to a SharePoint Online or OneDrive for Business site at a rate of at least 1 GB per hour.

After I upload my files to the Azure storage area, how long are they kept in Azure before they’re deleted?

When you use the network upload method to import SharePoint files, you upload the data files and the manifest files to Azure blob containers. The data and manifest files in these containers are deleted 45 days after the most recent import job was created.

Are document properties, such as title, author, modified date and other properties, preserved when documents are imported to Office 365?

Yes. None of the original document metadata is changed during the import process.

Using drive shipping to import SharePoint data

For step-by-step instructions, see Use drive shipping to import SharePoint data to Office 365.

Where is drive shipping available?

Drive shipping is currently available in the United States, Europe, India, East Asia, Southeast Asia, Japan, and Australia. Drive shipping will be available in more regions soon.

What commercial licensing agreements support drive shipping?

Drive shipping to import SharePoint data to Office 365 is available through a Microsoft Enterprise Agreement (EA). Drive shipping isn’t available through a Microsoft Products and Services Agreement (MPSA).

What is the pricing for using drive shipping to import SharePoint data to Office 365?

The cost to use drive shipping to import SharePoint data to Office 365 is $2 USD per GB of data. For example, if you ship a hard drive that contains 1,000 GB (1 TB) of data files, the cost is $2,000 USD. You can work with a partner to pay the import fee. For information about finding a partner, see Find your Office 365 partner or reseller.

What kind of hard drives are supported for drive shipping?

Only 2.5 inch solid-state drives (SSDs) or 2.5 or 3.5 inch SATA II/III internal hard drives are supported for use with the Office 365 Import service. You can use hard drives up to 10 TB. For import jobs, only the first data volume on the hard drive will be processed. The data volume must be formatted with NTFS. When copying data to a hard drive, you can attach it directly using a 2.5 inch SSD or 2.5 or 3.5 inch SATA II/III connector or you can attach it externally using an external 2.5 inch SSD or 2.5 or 3.5 inch SATA II/III USB adaptor.

IMPORTANT: External hard drives that come with an built-in USB adaptor aren’t supported by the Office 365 Import service. Additionally, the disk inside the casing of an external hard drive can’t be used. Please don’t ship external hard drives.

How many hard drives can I ship for a single import job?

You can ship a maximum of 10 hard drives for a single import job.

After I ship my hard drive, how long does it take to get to the Microsoft data center?

That depends on a few things, such as your proximity to the Microsoft data center and what kind of shipping option you used to ship your hard drive (such as, next-day delivery, two-day delivery, or ground-delivery). With most shippers, you can use the tracking number to track the status of your delivery.

How long does it take to import data to a SharePoint Online or OneDrive for Business site?

After your hard drive is received at the Microsoft data center, it’s typically processed within one business day. It takes several hours for each terabyte (TB) of data on the hard drive to be uploaded to the Microsoft Azure storage area for your organization. After the files are copied to the Azure storage area, data is imported to a site at a rate of at least 1 GB per hour. If different data files are imported to different sites, the import process occurs in parallel; in other words, each migration package/site pair is imported simultaneously. However, if migration packages are imported to the same target site, each migration package is queued and imported to the site one at a time.

After Microsoft uploads my files to the Azure storage area, how long are they kept in Azure before they’re deleted?

After we receive your hard drive, Microsoft data center personnel will upload your data and manifest files to Azure blob containers. The data and manifest files in these containers are deleted 45 days after the most recent import job is created.

Are document properties, such as title, author, modified date and other properties, preserved when documents are imported to Office 365?

Yes. None of the original document metadata is changed during the import process.

Can Microsoft wipe my hard drive before they ship it back to me?

No, Microsoft can’t wipe hard drives before shipping them back to customers. Hard drives are returned to you in the same state they were in when they were received by Microsoft.

Can Microsoft shred my hard drive instead of shipping it back to me?

No, Microsoft can’t destroy your hard drive. Hard drives are returned to you in the same state they were in when they were received by Microsoft.

What courier services are supported for return shipping?

If you’re a customer in the United States or Europe, Microsoft uses FedEx to return your hard drive. For all other regions, Microsoft uses DHL.

What are the return shipping costs?

Return shipping costs vary, depending on your proximity to the Microsoft data center that you shipped your hard drive to. Microsoft will bill your FedEx or DHL account to return your hard drive. The cost of return shipping is your responsibility.

Can I use a custom courier shipping service, such as FedEx Custom Shipping, to ship my hard drive to Microsoft?

Yes.

If I have to ship my hard drive to another country, is there anything I need to do?

The hard drive that you ship to Microsoft might have to cross international borders. If this is the case, you’re responsible for ensuring that the hard drive and the data it contains are imported and/or exported in accordance with the applicable laws. Before shipping a hard drive, check with your advisors to verify that your drive and data can legally be shipped to the specified Microsoft data center. This will help to ensure that it reaches Microsoft in a timely manner.

 

Read More

Overview of SharePoint 2013 Multibox installation and configuration

Here is a list of setup and correct procedural instructions for creating SharePoint 2013 three-tier farm.

For most of these steps I used powershell scripts.

1. Create Self-Signed SSL certificate or Order SSL certificate for URL domain(s)
2. configure hardware load balancing (if load balancing)
3. Provision physical or virtual servers – Windows 2012 (preferred) or Windows 2008 R2 SP1+
4. Install SSL certificate on servers
5. Create DNS entries for domains to resolve to Forefront/load balancer
6. Copy setup media to servers

  • Copy SharePoint 2013 setup files to SharePoint servers
  • Copy SharePoint 2013 Language Packs to SharePoint servers

7. Provision SQL Server 2014

  • Set Up SQL Server 2014 as a SharePoint 2013 Database Server
  • Configure SQL Server 2014 for SharePoint 2013
    • Setting the Maximum Amount of RAM
    • Enabling Compressed Backups
    • Setting the model Database’s Recovery Model
    • Configure the max degree of parallelism Server Configuration Option

8. Provision service accounts in Active Directory and grant permissions

  • Create Domain Accounts and Administration Groups
  • Assigned Group Policy
    • Grant SP Setup service account local Administrator on SharePoint servers
    • Grant SP Setup “Replicate Directory Changes” AD permissions
    • Grant SP Setup SQL Server permissions – DB Creator and Security Admin

9. Identify SharePoint 2013 and Office Web Apps 2013 product keys
10. Configure Kerberos Authentication

  • Register Service Principal Names (SPN)
  • Activate Kerberos Delegation for SharePoint Server in Active Directory
  • Activate Kerberos Authentication on Web Application in Central Admininstration

11. Identify outbound e-mail server details

  • An SMTP address for SharePoint outbound email (for alerts, etc.)
  • An e-mail address for the “From” or “Reply To” address in system e-mails

12. Configure Firewall Ports

13. Prepare the farm servers

  • Install SMTP Server
  • Add Local Groups and Domain Users to Groups
  • Configure Windows Server Firewall Ports

14. SQL/SharePoint Database Preparation

  • Configure SQL Database Performance

15. Install the following prerequisite software

  • .NET Framework 4.5
  • Windows Management Framework 3.0 (CTP2) – PowerShell 3.0
  • Microsoft SQL Server 2008 r2 Native Client
  • Windows Identity Foundation (KB974405)
  • Microsoft Sync Framework Runtime v1.0 SP1 (x64)
  • Windows Server AppFabric
  • Windows Identity Extensions
  • Microsoft Information Protection and Control Client
  • Microsoft WCF Data Services 5.0
  • CU Package 1 for Microsoft AppFabric 1.1 for Windows Server (KB2671…

16. Install SharePoint 2013 media files on ALL the farm servers

17. Perform A Windows Update

18. Reboot Windows 2012 Server

19. Create New Farm – (Run on First Application Server)

  • 7.1 Create the SharePoint Administration & Configuration content database.
  • 7.2 Add Administrators to Farm
  • 7.3 Create Managed Accounts in SharePoint
  • 7.4 Configure People Picker

20. Install SQL Add-ons

  • 8.1 Install SharePoint Reporting Add-on for SharePoint
  • 8.2 Install SharePoint PowerPivort Add-on for SharePoint

 

21. Run SharePoint Configuration Wizard on each server

22. Join Web Server to Farm

  • Install Web Servers
  • Install Services
  • Caims to Windows Token Service
  • App Management Service
  • Business Data Connectivity Service
  • Microsoft SharePoint Foundation Workflow Timer Service
  • Microsoft SharePoint Foundation Incoming E-Mail
  • Microsoft SharePoint Foundation Web Application
  • Secure Store Service
  • Microsoft SharePoint Foundation Subscription Settings Service
  • Managed Metadata Web Service
  • Search Host Controller Service
  • Search Query and Site Settings Service

23. Install Applications

  • Secure Store Service
  • Subscription Settings
  • App Management Service
  • Managed Metadata Service
  • Create Default Web Application
  • Search Service Application

24. Business Data Connectivity Service

25. Restore Site Collection

26. Install Language Packs

27. Join Application Server to Far

28. Install Application Server

  • Install Services
    • Distributed Cache
    • Machine Translation Service
    • Word Automation Services
    • Work Management Service
    • PowerPoint Conversion Service
    • PerformancePoint Service
    • SQL Server PowerPivot System Service
  • Install SharePoint Applications
    • Usage and Health Data Collection
    • Machine Translation Service
    • State Service Application
    • Word Automation Services
    • Work Management Service Application
    • PerformancePoint Service Application
    • Excel Services Application
    • Visio Graphics Service

29. Run SharePoint Configuration Wizard

30. Configure Secure Store

31. User Profile Synchronization

32. Configure email integration for a SharePoint 2013 farm

  • Configure incoming email for a SharePoint 2013 farm

33. Configure outgoing email for a SharePoint 2013 farm

34. Run SharePoint Configuration Wizard

35. Perform A Windows Update

Read More

The Top Three Content Migration Tools for SharePoint

Over past several years, I have been involved in a projects migration SharePoint content, legacy systems and file shares to SharePoint. Enterprise level migration from SharePoint 2003 to SharePoint 2013.

Many clients determine that the best approach to complete the migration in a manner that conformed to their practices and procedures they wanted to purchase a third party tool.
Many tools were considered. In the end, the list of options came down to three tools. I have compared three products with SharePoint OOTB and with those functions within each product. Each product has been looked into in detail, over 170 sub functions, and are highlighted in the table below.

FunctionSharePoint OOTB (%)AvePoint (%)Metalogix (%)Sharegate (%)
High Level Features66.67100100100
Farm Migration0100100100
Site Migration9.0990.91100100
Content Migration0100100100
Import files to SharePoint0100100100
Export From SharePoint075100100
Reporting085.7110085.71
Powersell and Development tools501007575
Supported SP Legacy Versions0100100100
Support0100100100
Backup11.1194.4483.3333.33
RBLOB Management16.6710091.670
End User Content Management00100100
Office 365 Migration/Management0010063.64
OneDrive Migration/Management0071.4385.71
Overall (%)9.6071.6388.8459.23

AvePoint – DocAve Migrators
AvePoint is the established leader in enabling enterprise collaboration across platforms and devices. Focusing on helping enterprises in their digitization journey to enable their information workers to collaborate with confidence, AvePoint is first to market with a unique solution that centralizes access and control of information assets residing in disparate collaboration and document management systems on-premises and in the cloud. AvePoint solutions and services aim to bring together business, IT, as well as compliance and risk officers to serve key business objectives such as big data, cloud integration, compliance, enterprise content management, and mobile data access monitoring.
New Features

End-User Reporting – DocAve Report Center includes new end-user reporting web parts, allowing easy access to information such as site traffic, search usage, active users, checked-out documents, and top documents in order to help business users make more informed decisions.

Compliance and Governance Reporting – Usage Pattern Alerting for DocAve Report Center provides SharePoint administrators as well as security and compliance officers with new reports to quickly identify suspicious user activity in order to take actions that can help reduce the potential of data breaches.

Office 365 Support for Records Management – DocAve Archiver now offers customized records management solutions for both on-premises and SharePoint Online deployments – including new features to retain a document’s original content types and metadata properties upon moving to Record Center sites as well as the ability for administrators to set automated conflict resolution rules when archiving content.

Content Archiving Approval – Archiver Approval Center allows administrators to generate reports that empower end-users to review content before archiving takes place, allowing those closest to the content to determine whether or not it should be archived.

Office 365 Support for Deployment Management – DocAve Deployment Manager now supports Office 365 as well as hybrid infrastructures, enabling organizations to quickly deploy customizations and design elements such as apps between SharePoint 2013 on-premises and SharePoint Online environments.
Virtual Machine Backups: DocAve Backup and Restore combines its best-of-breed platform backup solution with new ability to protect virtualized elements such as web front end servers, Central Administration, and app servers.

Existing Features

  • Security Trimming
  • Source Content Pre-scan
  • Customizable And Reusable Migration Plans
  • Full Or Incremental Migration
  • Migration With Full Fidelity
  • Source And Destination Environment Coexistence During Migration
  • Online Or Offline Migration
  • Data Synchronization With Customized Scheduling

AvePoint is a well established player in the migration market these days. There tools are excellent but, for many customers, that software requires being installed on the production environments was a major issue and most declined this product for this reason. ________________________________________

Metalogix – Content Matrix
The most comprehensive tool on the market. Content Matrix allows you to migrate directly to SharePoint 2013, OneDrive for Business or Office 365 from SharePoint 2007 or SharePoint 2010 in one hop. No need for intermediate SharePoint versions or an on-premises staging farm for a move to Office 365.
Selected by Microsoft as the only pre-approved, MSO-CAF certified SharePoint migration tool for migrating to Office 365 Dedicated.

Features

  • Simple Setup
  • Rearrange Post-Migration
  • File Share Pre-migration tools
  • No Server Side Install
  • Migrate Discrete Elements
  • Powerful File Migration Capabilities
  • Choose What Migrates
  • Create New Structure
  • Analyze Metadata and Filter What You Need
  • Keep Current Farm Running
  • Migrate to 2013 MySites
  • Externalize BLOBS Before Migration
  • Site Collection Migration
  • Split/Merge Lists
  • Migrate Nintex Workflows
  • Clean Before Migration
  • Reusable Migration Tasks
  • Advanced Analysis & Migration Capabilities
  • Extensible Migration Engine

Sharegate – SharePoint Migration Tool

ShareGate Migration one of the most popular migration tools one of the reasons for this are its wonderful nicely design interface and it’s price. Single user license are around £2900 and is for migrating data only.

Here, is a list of the most common and interesting features available with ShareGate.

Copy Site Objects
This is a feature for migrating SharePoint site objects from one SharePoint site to another. With a simple drag and drop, it’s possible to copy site collections, sites, lists, libraries, site columns, site content types, users, groups, permission levels, managed metadata and workflows. Choose what you want to copy and let this feature do all the work.

Copy SharePoint Content
This feature can be used for copying SharePoint list items as well as SharePoint documents between lists or libraries. With a simple Drag and Drop, you can migrate your content while bringing over all metadata, including version history, attachments, permissions, authors and timestamps.

Import Files to SharePoint – Use this feature to import from your file system or file shares to SharePoint. With a simple drag and drop, it is possible to migrate a complete folder hierarchy to SharePoint while applying content types and metadata at the same time.

Export from SharePoint – This feature can be used for archiving content from SharePoint lists, libraries and sites. With a simple drag and drop, you can export all your content while bringing over all the metadata, including version history and attachments.

Bulk Metadata Editor – This feature provides bulk editing capabilities on SharePoint lists and libraries. In just a few clicks, it is possible to massively edit the SharePoint metadata of over thousands of documents, folders, document sets and list items without having to modify their properties one by one. You will find more details here.

Import from Google Drive – Use this feature to import from your Google Drive environment. With a simple Drag and Drop, you can migrate your content while bringing over all metadata, including version history, permissions, author information and timestamps.

Read More

Health Analyzer – Secondary Logon service (seclogon) is disabled

Summary: The Secondary Logon service is used to generate thumbnail images of Power Pivot workbooks in the Power Pivot Gallery. By default, the Secondary Logon service is set to manual startup.

If the service is disabled, thumbnail generation will fail. Additionally, the ULS logs will contain the following error: “The error 1058 can have as a root cause the fact the Windows service “Secondary Logon” is disabled.”

To check service configuration, use the Services console application to find Secondary Logon and change its Startup Type to Manual,  but the Health Rule checks if the Secondary Logon service has a Status of Running, not a Startup Type of Manual (or Automatic), so you can change this to Running.

Secondary Logon Properties

If you cannot enable the service, your organization might have a group policy that disables it. Check with an administrator to determine whether this is the case.

After you enable the service, thumbnail or snapshot images will refresh over time. Optionally, you can force a refresh by restarting the service and opening and then resaving the property pages of a specific report.

 

Read More

Health Analyzer – Verify that OAuth is configured correctly for the Machine Translation Service application proxy

Applies to: SharePoint Server 2013, SharePoint Foundation 2013

Summary: Resolve the SharePoint Health Analyzer rule “Verify that OAuth is configured correctly for the Machine Translation Service application proxy.”

Rule Name:  Verify that OAuth is configured correctly for the Machine Translation Service application proxy.

Cause:  OAuth is not configured correctly for the Machine Translation Service application proxy.

Resolution: Ensure that every Web application with a Machine Translation Service application proxy has a connection to a User Profile service application and an App Management service application, and is in claims-based authentication mode.

  1. Verify that the user account that is performing this procedure is a member of the Farm Administrators group.
  2. On the Central Administration website, clickApplication Management.
  3. On the Application Management page, in theService Applications section, click Configure service application associations.
  4. In theApplication Proxy Group column, click the proxy group for the Web application or service application that you want to configure. Usually it is the default Application Proxy Group.
  5. Select theUser Profile Service Application Proxy check box and the App Management Service Application Proxy check box.
  6. Go back to the Central Administration Home page. In theApplication Management section, click Manage web applications.
  7. Click the Web application you want to configure, and then click theAuthentication Providers button on the ribbon.
    1. Ensure that the Membership Provider Name for theDefault zone is Claims Based Authentication. If not, you have to migrate the Web applications from classic mode to claims-based authentication. For more information, see Migrate from classic-mode to claims-based authentication in SharePoint 2013.
Read More

User Profile Synchronization Errors In SharePoint 2013

Applies to: SharePoint Server 2013, SharePoint Foundation 2013

Summary: Permission and the errors when creating the user profile synchronising job of SharePoint 2013.

During testing of SharePoint 2013 I experienced the Application Event Log errors below.  When the errors occurred I was unable to run a User Profile Synchronization with Active Directory.

Event ID 6398, category Timer

The Execute method of job definition Microsoft.Office.Server.UserProfiles.UserProfileImportJob (ID 0afab701-a201-4df9-bfc7-590838da8809) threw an exception. More information is included below.

Generic Failure

Event ID 1004, category None

Detection of product ‘{90150000-104C-0000-1000-0000000FF1CE}’, feature ‘PeopleILM’, component ‘{1C12B6E6-898C-4D58-9774-AAAFBDFE273C}’ failed. The resource ‘C:Program FilesMicrosoft Office Servers15.0ServiceMicrosoft.ResourceManagement.Service.exe’ does not exist.

Event ID 1001, category None

Detection of product ‘{90150000-104C-0000-1000-0000000FF1CE}’, feature ‘PeopleILM’ failed during request for component ‘{1681AE41-ADA8-4B70-BC11-98A5A4EDD046}’

Resolution

The resolution is to grant read access to the Network Service account to the c:Program FilesMicrosoft Office Servers15.0 folder

Read More

Configuring the Windows Firewall for SharePoint Farm Traffic (Muti-Tier)

Applies to: SharePoint Server 2013, SharePoint Foundation 2013

This document outlines the firewall ports required to be opened and on which server depending on the services running (Windows, SharePoint, Database Server) for a mutli-tiered SharePoint Server farm.

Of course, this is only one scenario and it all depends on SharePoint farm and network configuration you have and whether you configured behind gateways, firewalls etc.

SharePoint Network Diagram – An example

SharePoint Farm Firewall Ports

Web Front End Server
When a range is specified all ports between the range must be opened.

PortsProtocolBoundUsage
22233-22236TCPOutPorts required for the AppFabric Caching Service
80TCPInhttp
443TCPInhttps/ssl
25TCPInSMTP for e-mail integration
16500 – 16519TCPInPorts used by the search index component
1433TCPOutServer default communication port (if no alias or custom port)
1434UDPOutSQL Server default port used to establish connection (if no alias or custom port)

Distributed Cache
When a range is specified all ports between the range must be opened.

PortsProtocolBoundUsage
22233-22236TCPIn/OutDistributed Cache

Application Server
When a range is specified all ports between the range must be opened.

PortsProtocolBoundUsage
22233-22236TCPOutPorts required for the AppFabric Caching Service
80TCPInClient to SharePoint web server traffic (SharePoint – Office Web Apps communication)
443TCPInEncrypted client to SharePoint web server traffic (Encrypted SharePoint – Office Web Apps communication)
CustomTCPInSharePoint Central Administration v4
25TCPInSMTP for e-mail integration
16500 – 16519TCPInPorts used by the search index component
32843-32845TCPInCommunication between Web servers and service applications Inbound rule Added to Windows firewall by SharePoint
32846TCPIn/OutSharePoint User Code Service Inbound rule Added to Windows firewall by SharePoint
808-809TCPInOffice Web Apps

Search Index Query Server
When a range is specified all ports between the range must be opened.

PortsProtocolBoundUsage
22233-22236TCPOutPorts required for the AppFabric Caching Service
16500 – 16519TCPInPorts used by the search index component
137 - 139TCPOutSMB – Index Propagation / File Shares crawl ( TCP )
445TCP / UDPInSMB – Index Propagation / File Shares crawl ( NetBIOS )

Search Administration & Crawl Content Processing
When a range is specified all ports between the range must be opened.

PortsProtocolBoundUsage
22233-22236TCP Outhttp
80TCP Outhttps/ssl
443TCP Out
16500 – 16519 TCPIn / OutPorts used by the search index component
137 - 139TCPOutSMB – Index Propagation / File Shares crawl ( TCP )
445TCP / UDPOutSMB – Index Propagation / File Shares crawl ( NetBIOS )
1433TCPOutServer default communication port (if no alias or custom port)
32843-32845TCPIn Communication between Web servers and service applications Inbound rule Added to Windows firewall by SharePoint

Routes to the to query server
The crawl component processes crawls of content resources, and propagates the resulting index fragment files to query server components:
• TCP Port 32845 (SML/Named Pipes)

Routes to the Webserver
Windows Communication Foundation (WCF)
• TCP port 32843
• TCP port 32844 (SSL)

Routes to the Database server
Database communication:
On the Query Server, the query processor (also known as the Search Query and Settings Service) communicates with the following two databases in SQL Server:

• Search Administration database
• Property database types

On the Crawl Server, each crawl component is attached to a crawl database in SQL Server. The crawl component adds information such as content resource location and crawl schedules to its associated crawl database.

• TCP/SSL port 1433 (default) for default instance (customizable)
• TCP/SSL random port for named instances (customizable)

Routes to the Webserver
Search crawling —The crawl component on the Crawl Server processes crawls of content resources. Depending on how authentication is configured, SharePoint sites might be extended with an additional zone or Internet Information Services (IIS) site to ensure that the index component can access content; this configuration can result in custom ports.
• TCP 80
• TCP 443 (SSL)
• Custom ports

Active Directory Server

The following table lists the port requirements for inbound connections from each server role to an Active Directory Domain Services domain controller.

PortsProtocolBoundUsage
88TCP / UDPInUser Profile Synchronization Service(FIM) (Kerberos)
445TCP / UDPInWhen SQL Server is configured to listen for incoming client connections by using named pipes over a NetBIOS session, SQL Server communicates over TCP port 445
389TCP / UDPInUser Profile Synchronization Service(FIM) (LDAP )
464TCP / UDPInUser Profile Service(FIM) -User List Resolution / Kerberos password change
5725TCPInUser Profile Synchronization Service(FIM) - Synchronizing profiles between SharePoint 2013 and Active Directory Domain Services (AD DS) on the server that runs the Forefront Identity Management agent

LDAP/LDAPS ports are required for server roles based on the following conditions:

  • Web servers   Use LDAP/LDAPS ports if LDAP authentication is configured.
  • Query server   Role requires LDAP/LDAPS ports for importing profiles from the domain controllers that are configured as profile import sources, wherever these reside.

Names Resolution (DNS) Server

The following table lists the port requirements for inbound connections from each server role to a Domain Name System (DNS) server. In many extranet environments, one server computer hosts both the Active Directory Domain Services domain controller and the DNS server.

PortsProtocolBoundUsage
53TCP / UDPInUser Profile Synchronization Service(FIM) - DNS

Distributed Cache Server

SQL Server

The following table lists the port requirements for inbound connections from each server role to a server running SQL Server. Most database connections are on TCP port 1433 by default for the default instance, but this is customizable. (Because TCP port 1433 is a well-known port for accessing a SQL Server, you could configure your server running SQL Server to listen on a different port and block port 1433.) For named instances, you can choose a random port.

PortsProtocolBoundUsage
1433TCPInServer default communication port (if no alias or custom port)
1434UDPInSQL Server default port used to establish connection (if no alias or custom port)
445TCPInSQL Server using named pipes
2383TCPInSQL Analysis Server default communication port (if no alias or custom port)

External Content File Share Server

PortsProtocolBoundUsage
137TCP InSMB – Index Propagation / File Shares crawl ( TCP )
138TCP InSMB – Index Propagation / File Shares crawl ( TCP )
139TCP InSMB – Index Propagation / File Shares crawl ( TCP )
445TCP / UDPInSMB – Index Propagation / File Shares crawl ( NetBIOS )

External Content (OLAP) Server

PortProtocolBoundUsage
1433TCPInSQL (Default Instance)
1434UDPInSQL (Default Instance)

SMTP Server

E-mail integration requires the use of the Simple Mail Transport Protocol (SMTP) service using TCP port 25 on at least one of the front-end Web servers in the server farm. The SMTP service is required for incoming e-mail (inbound connections). For outgoing e-mail, you can either use the SMTP service or route outgoing e-mail through a dedicated e-mail server in your organization, such as a computer running Microsoft Exchange Server.

PortsProtocolBoundUsage
25TCPInSMTP - Cannot be configured

Read More
1 2 3