Tuesday, April 14, 2015

AADSync Performance Counters Error

While working on a project recently, I came across the following error on my AADSync server:

Log Name:      Application
Source:        ADSync
Date:          1/12/2015 12:47:11 PM
Event ID:      6313
Task Category: Server
Level:         Error
Keywords:      Classic
User:          N/A
Computer:      AADSync.contoso.com
Description: The server encountered an unexpected error creating performance counters for management agent “xxxxx.onmicrosoft.com – AAD”.

Performance counters will not be available for this management agent.

To fix this issue in AADSync, you can either perform a clean install (often out of the question) or run the following commands to reload the performance counters:

  1. Stop AADSync’s service;
  2. Delete the following registry key: [HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\ADSync\Performance];
  3. Recreate the Performance key;
  4. Run the following two commands from an elevated command prompt:
    • unlodctr.exe ADSync
    • lodctr.exe “C:\Program Files\Microsoft Azure AD Sync\Bin\mmsperf.ini”
  5. Start AADSync’s service.

Wednesday, April 1, 2015

Exchange Online Protection Quarantine

A decade ago, Bill Gates predicted a spam-free world by 2006. Although we are seeing a small decline in spam, this is unfortunately far from coming true... Exchange Online Protection (EOP) does a great job, in my opinion, at filtering out obvious spam. According to the latest figures from Microsoft, ten million spam messages are blocked every single minute on average by EOP, 10 million! That is an impressive number. However, every day attackers around the world come up with new techniques to fool spam detection engines. Threats take different forms, such as an unidentified spam campaign, unknown malware or a completely new virus. This means that a small percentage (around 3%) of email that is likely to be spam still comes through and are sent to users’ Junk E-mail folder. Users obviously do not want spam in their inboxes, but they often have to review this folder to make sure no good messages (false positives) are mixed in with the bad.
EOP provides two main methods of handling spam detected by its content filters. Administrators can configure it so that spam is sent to the Junk E-mail folder in Outlook and Outlook Web App (OWA), which is the default option, or to direct it into a web-based quarantine.
Sending spam to the Junk folder is the most common choice as that is what users have been using for many years. But from experience I also noticed that this is the case as not everyone is aware of the quarantine feature. On the other hand, some customers have non-Exchange email systems that do not support the Junk E-mail folder approach, have a 3rd party filtering system that sends spam reports to users, or simply prefer the spam quarantine.
Since EOP was launched it has supported spam quarantine, but initially administrators were the only ones who had access to this quarantine, through the Exchange Admin Center, and only they were able to release spam messages... But for some time now administrators can configure EOP to give users self-service management of spam-quarantined messages. So let us have a look at how this works and how we can configure it.
In this article, we will explore the Quarantine feature of EOP, including how to enable, configure and manage it both from the administrator and end user perspectives. To continue reading, please go to the Exchange Online Protection Quarantine article at MSExchange.org.

Friday, March 27, 2015

Speeding up the Exchange Hybrid wizard in global deployments

If you ever ran the Exchange Hybrid wizard in an environment with servers all over the world, it is likely that it took a few hours to run. But why?
If we look at the wizard’s logs ($exinstall\Logging\Update-HybridConfiguration), we will see that most of the changes are fairly quick. However, it eventually goes on to run a Get-WebServicesVirtualDirectory to analyse the EWS virtual directories (VDs) across all Exchange servers in the environment to determine if any need to be configured. If this comes back true, then the wizard runs the same cmdlet again followed by a Set-WebServicesVirtualDirectory to enable the MRS Proxy for VDs that currently have it disabled. After all the necessary EWS VDs are configured, the wizard runs a Get-WebServicesVirtualDirectory for a third time to validate the configuration/changes made.
The problem here is running the *et-WebServicesVirtualDirectory cmdlet between servers in different countries or even continents. How long does it take for you? Usually it should be a few minutes for each server, but I have seen cases where it takes 30 minutes or more. Now multiply that by the total number of Exchange servers and it can quickly turn into hours and hours...
If, for example, your environment also has Exchange 2007 servers, although these do not use or have the MRS Proxy service, because the wizard simply runs the Get-WebServicesVirtualDirectory cmdlet, this returns 2007 servers (instead of filtering them out...), which contributes to delaying the process.
So, to speed things up a bit, you can manually login to all the servers, enable the MRS Proxy and only then run the Hybrid Wizard. Typically I was only enabling it on the Hybrid servers or servers that I was planning to use for mailbox migration, but the wizard enables it across estate anyway...
To recap, the Mailbox Replication Service Proxy (MRS Proxy) facilitates cross-forest mailbox moves and remote move migrations between an on-premises Exchange organization and Exchange Online. During cross-forest and remote move migrations (aka hybrid migrations), a Client Access server acts as a proxy for incoming move requests for the Mailbox server. The ability of a Client Access server to accept these requests is disabled by default. To allow the Client Access server to accept incoming move requests, we have to enable the MRS Proxy endpoint.

Monday, March 23, 2015

Azure Active Directory Connect Public Preview

The latest version of the Azure AD Connect has been released – the March 2015 Public Preview update.

The Azure AD Connect wizard Public Preview provides a guided experience for integrating one or multiple AD forests with Microsoft Azure AD. Optionally you can configure Exchange Hybrid deployment, password change write-back, ADFS and Web Application Proxy.

Azure AD Connect encompasses functionality that was previously released as DirSync and AADSync. These tools will eventually stop being released individually and all future improvements will be included in updates to Azure AD Connect.

This latest version has been updated with new capabilities, support for additional sync options, Additional Tasks and “Pilot Mode”.

You can download it from the Connect website.

Friday, March 20, 2015

How to change AADSync credentials

When it comes to changing the credentials AADSync uses to connect to the on-premises Active Directory (AD) or to Azure AD, one might think that re-running the wizard and updating the credentials there would do the trick:

However, if you re-run the wizard again, you will see that the old credentials are still being used... So how can we change these credentials?! To do this, we need to use the miisclient.

First, navigate to "install dir"\Microsoft Azure AD Sync\UIShell and run missclient.exe. Then, click in Connectors. Here you will have two connectors, one is used to connect to the local AD and the other to connect to Azure AD:
To update the credentials used to connect to the local AD, double-click the respective connector and then go to Configure Directory Partitions. Here, select Alternate credentials for this directory partition, enter the new credentials and click OK:
To update the credentials used to connect to Azure AD, double-click the respective connector and then go to Connectivity. Here, enter the new credentials and click OK:
Job done!

Thursday, March 19, 2015

Dynamic Distribution Lists in a Hybrid Environment

In a hybrid deployment environment between Exchange Online and on-premises Exchange organizations, neither DirSync nor AADSync can be used to synchronize dynamic distribution lists (DDL) to Exchange Online. Therefore, mailboxes that have been migrated to Exchange Online cannot see DDLs in their GAL or email them.
To work around this issue, create a MailContact in Exchange Online for the DDL, and then grant permissions so that only authenticated senders can submit messages to the new contact. This object should have the following mappings:
On-Premises DDL
Cloud MailContact
You should also consider the scope of the DDL before mailboxes are moved to Exchange Online. If the scope included only mailboxes, the scope must be expanded to include mail users and mail contacts. To do this, open the properties of the DDL and on the membership tab (in Exchange 2013), also select Mail users with external email addresses. If using the Shell, add MailUsers to the IncludedRecipients property of the DDL.
Exchange Online users can now select the DDL from the GAL. When they do, messages will be delivered to the members of the group as defined by the settings for the group.

Friday, March 13, 2015

Exchange Online Protection Conditional Mail Routing

Simply put, Conditional Mail Routing, also known as Criteria Based Routing, is a way of configuring Exchange Online Protection [EOP] connectors in order to send or receive mail a certain way based on the condition of the individual email. For example, we can force TLS for a specific sender or route email based on recipients’ properties to different email server locations.
I was recently working on a global Exchange migration to Office 365. This particular client had an Office 365 tenant hosted in Europe with a Hybrid deployment in India. On top of that, there was a separate Exchange organization deployed across the United Kingdom and the United States of America.
For us to be able to update the MX records to point to the client’s Office 365 tenant, two requirements had to be met because of security and legal reasons:
  • Requirement 1: emails sent from partners to certain business application mailboxes (hosted on-premises in the UK) had to go directly from Office 365 to the Exchange servers in the UK, i.e., without being routed through India;
  • Requirement 2: emails addressed to US users had to go directly from Office 365 to the Exchange server on-premises in the US, i.e., without being routed through India.

To see how we can meet these 2 requirements using Conditional Mail Routing, please check my Exchange Online Protection Conditional Mail Routing article at MSExchange.org.

Sunday, February 22, 2015

Exchange Online Accept mail for all subdomains feature

In Exchange Online (not on-premises), under your Accepted Domains, you might have seen an option to accept mail for all subdomains:
When this feature is enabled for a domain, emails can be sent and received for subdomains of this domain. For example, if nunomota.pt is a provisioned domain and match subdomains support is enabled, users can send emails to or receive emails from a.nunomota.pt, b.nunomota.pt, a.b.nunomota.pt, and other subdomains.
This feature is usually for customers who have a hybrid environment with mailboxes that reside on-premises as it is only applicable for the Internal Relay domain type.
Following the example above, once the feature is enabled for the domain, Office 365 will be able to deliver emails that are sent to @a.b.nunomota.pt addresses by automatically forwarding them to my on-premises environment (assuming all connectors are in place).
But there is a small catch! Spam. Emails will not get blocked by EOP, meaning spammers can send millions of invalid emails to random addresses and their subdomains in order to try to overwhelm on-premises servers. Having said that, this is the case with most relay scenarios anyway.

Friday, January 23, 2015

Exchange Online Onboarding Message Size Limit Now 150 MB

Whenever administrators migrate mailboxes to Exchange Online, an onboarding limit in terms of message size always apply in order to avoid migrating huge messages and potentially impact service. Up until now this limit was 25MB (attachment size) plus overhead, so approximately 36MB in total. If a mailbox contained any emails over this size, they would simply be skipped and administrators had to find alternative ways to import them. To be honest, not many companied allow such big emails, but I have worked with a few that do, which would cause an issue if migrating to Office 365...

Now, this limit has been increased to 150MB (and is already available to all customers)! However, please have the following in mind:
  • This new limit only applies to onboarding moves using the native Mailbox Replication Service (MRS) and which target AD user objects with a RecipientType of MailUser (MEU) that do not have MaxReceiveSize stamped. For example, hybrid deployments;
  • All other data migration solutions, both using Microsoft tools and 3rd party tools target UserMailbox (MBX) AD users (i.e., rely on the mailbox already being created in Exchange Online). The increased onboarding message size limit does not apply to these solutions which will use the limit (MaxReceiveSize) configured on these MBX objects. The maximum-allowed onboarding message size limit for these solutions remains 36MB. This includes merges (staged and cutover migration), IMAP, EWS, MAPI, 3rd party tools, and PST Import.

The good news is that soon administrators will be able to customize the send/receive size limit, enabling them to modify the MaxSendSize and MaxReceiveSize limits on their MBXs and MEUs, effectively overcoming this limitations (hopefully).

The Exchange Online Service Description has now been updated to reflect this change.

Wednesday, January 21, 2015

Managing Exchange Online using Server 2012 R2 Essentials Experience Role

Windows Server 2012 Essentials and the Essentials Experience role build on the previous Office 365 Integration Module for Small Business Server 2011 Essentials. This option is now part of the core product (not a separate download) and provides a seamlessly integrated management experience on Essentials for customers who are using Exchange Online.

On top of the core feature set that was included in the Office 365 Integration Module for SBS 2011 Essentials, such as integrated user account management and automatic user password synchronization, Microsoft also made a few enhancements to make the experience better:
  • Support for multiple e-mail addresses. Having multiple domains and/or assigning multiple e-mail addresses to a single user are common scenarios even for small businesses. Now it is finally possible to easily do that from within the Essentials’ Dashboard;
  • Improved Office 365 domain configuration wizard. In the Office 365 Integration Module for SBS 2011 Essentials, administrators were required to configure Remote Web Access (RWA) when configuring a domain for Office 365, which caused a lot of confusion. For example, administrators had to provide an SSL certificate which was not actually needed by Office 365, but was required by RWA. Now these two have been de-coupled. Another improvement is the option to configure a different domain name for Office 365 and for RWA, allowing small businesses to continue to use the same free domain names like letsexchange.remotewebaccess.com for RWA on Essentials and a different domain name for e-mail in Office 365;
  • Display mailbox usage information. The Office 365 tab on the Essentials Dashboard now shows the mailbox usage information.

To read the full article, please check my Managing Exchange Online using Server 2012 R2 Essentials Experience Role article series on MSExchange.org.

Monday, December 15, 2014

Removing a Domain from Office 365 – What to check for

To remove a domain from Office 365, make sure that no settings are using the domain. You will not be able to remove the domain if one or more of the following conditions are true:
  1. User accounts or groups are associated with the domain;
  2. The proxies that correspond to the domain for all mail-licensed users and for all mail-enabled groups are not removed. Office 365 blocks the deletion of a domain until the proxies that correspond to the domain are removed;
  3. Lync Online Session Initiation Protocol addresses are used by the domain.
This is mostly common knowledge and what I usually check for when removing a domain. However, the last time I did this in one of my test tenants, I just wasn’t being able to remove it...
Initially I used the GUI:
But it always got stuck in this window no matter how long I waited for:
So I decided to do it from the Shell. After connecting to Office 365 I tried using the Remove-MsolDomain cmdlet with not much success:
So I checked for users associated with the domain I was trying to remove and there were none. By I forgot I had created contacts before, so I checked for any Exchange Online object that contained at least one email address that matched the domain, and voila!
Ok, certainly now it had to work, right? Not really...
Another inspiration and I remembered I had created an outbound connector for this domain to test Exchange Online Protection Conditional Mail Routing! And here it was:
So I removed it using the Shell and finally I was able to remove the domain!
Bottom line is, check everything! Including:
  • Whether user names contain the domain name: Get-MsolUser -DomainName “yourdomain.com”;
  • All recipients’ email addresses: Get-Recipient | Where {$_.EmailAddresses -match “yourdomain.com”};
  • Transport Rules;
  • Connectors;
  • Your public website hosted on Office 365.

Monday, December 8, 2014

Exchange 2013 Queue Velocity

When you use the Get-Queue cmdlet in Exchange 2013 you will see a Velocity property (not visible in the Queue Viewer tool):

So what exactly is this Velocity? Does zero means no emails is coming in or going out? Is zero a good thing?

The Velocity property is simply the drain rate of the queue. Exchange 2013 measures the rate of messages entering and leaving every queue and stores these values in queue properties. These rates can be used as an indicator of queue and transport server health. There are three properties: Velocity (which we have seen in the previous screenshot) and IncomingRate and OutgoingRate, both visible in the screenshot just below:

Their meaning is as follows:

This values is the rate at which messages are entering the queue. It is calculated from the number of messages entering the queue every 5 seconds averaged over the last 60 seconds. The formula can be expressed as (i1+i2+i3+i4+i5+i6)/6, where in is the number of incoming messages in 5 seconds.

Let us say that, as an example, we received 12 messages in the first 5 seconds and then 8 in the following 5 seconds. As such, our incoming rate is (12+8)/6 = 3.333

This value is the rate that messages are leaving the queue. It is calculated from the number of messages leaving the queue every 5 seconds averaged over the last 60 seconds. The formula can be expressed as (o1+o2+o3+o4+o5+o6)/6, where on is the number of outgoing messages in 5 seconds.

Continuing with our previous example, let us say that in the first 5 seconds 4 messages were sent, followed by 7 messages in the following 5 seconds and 9 messages in the next 5 seconds. As such, our outgoing rate is (4+7+9)/6 = 3.333

This property is the drain rate of the queue, and is calculated by subtracting the value of IncomingRate from the value of OutgoingRate.

In our example Velocity = OutgoingRate – IncomingRate = 3.333 – 3.333 = 0.
Although messages took slightly over to leave the queue (5 extra seconds), remember that what is considered is the average over the last 60 seconds, in which case, messages left the queue at the same time they entered in our example.

  • If the value of Velocity is greater than 0, messages are leaving the queue faster than they are entering the queue.
  • If the value of Velocity is equals 0, messages are leaving the queue as fast as they are entering the queue. This is also the value we see when the queue is inactive.
  • If the value of Velocity is less than 0, messages are entering the queue faster than they are leaving the queue, which is not ideal.

Bottom line: a positive value of Velocity indicates a healthy queue that is efficiently draining, and a negative value of Velocity indicates a queue that is not efficiently draining.

Does this mean I need to worry whenever this value is not zero? Well, not exactly. We also need to consider the values of the IncomingRate, OutgoingRate, and MessageCount properties, as well as the magnitude of the Velocity value for the queue. If someone all of the sudden sends a large email to everyone in your organization, it is possible that the Velocity will be negative for a short while.

Monday, December 1, 2014

Attributes Synchronized to Azure AD by AADSync

If you want to know exactly what Active Directory (AD) attributes get synchronized to Azure AD by AADSync, or which AD attributes each Office 365 service consumes, the tables in this webpage will provide you with all the information you need!

Thursday, November 27, 2014

Exchange Online Shared Mailboxes Limit Now 50GB!

Although these limits have been in effect for some time now, it is good to finally have an official confirmation that Shared Mailboxes in Office 365 now also have 50GB of mailbox storage available!

Monday, November 24, 2014

DirSync vs AADSync

The eventual successor to Azure Active Directory Synchronization Tool (DirSync) is the Azure Active Directory Synchronization Service, also known as AADSync. Although AADSync provides new features that DirSync does not, it also lacks a few features currently in DirSync, especially in the first GA version... So what exactly are the differences between the two?!

For a comparison chart of the features that each of these tools currently supports for synchronizing your directory with Azure Active Directory, visit the Directory Integration Tools webpage.

Monday, November 17, 2014

Exchange 2013 OWA “The following files couldn't be attached” error

If are trying to add a large attachment to an email in Outlook Web App (OWA) you might get one of the following errors:
• The following files weren't attached because adding them would cause the message to exceed the maximum size limit of 10 MB: filename.
• The following files couldn't be attached: Filename. Please try again later.

As you can see from the first error, the problem is self-explanatory: the attachment is just too big. In Exchange 2013 the default maximum message size for an attachment is 10 megabytes (MB).

To Increase this limit you need to follow these steps:
  1. Increase the maximum message size for the organization by using the Set-TransportConfig cmdlet;
  2. Increase the maximum message size for the Send connectors by using the Set-SendConnector cmdlet;
  3. Increase the maximum message size for the Receive connectors using the Set-ReceiveConnector cmdlet;
  4. Increase the following settings in the OWA Web.config file:
    • maxAllowedContentLength (value in bytes);
    • maxReceivedMessageSize (value in bytes);
    • maxStringContentLength (value in bytes);
    • maxRequestLength (value in kilobytes).
  5. Increase the following settings in the EWS web.config file:
    • maxAllowedContentLength (value in bytes);
    • maxReceivedMessageSize (value in bytes).
  6. Stop and then restart the MSExchangeOWAAppPool application pool;
  7. Stop and then restart the MSExchangeServicesAppPool application pool.

IMPORTANT: due to the new architecture of Exchange 2013, steps 4 and 5 must be performed in both of the following locations:
  • The Client Access server on which the Web.config files are located in the following path: %ExchangeInstallPath%\FrontEnd\HttpProxy
  • The Mailbox server on which the Web.config files are located in the following path: %ExchangeInstallPath%\ClientAccess

Also, in step 4, please note that not all settings are present on both files.

Monday, November 10, 2014

AADSync ProxyAddresses Not Synchronized to Office 365

If you have installed the first publicly available version of AADSync (v1.0.0419.0911), the eventual successor to DirSync, you might have noticed that the ProxyAddresses attribute will not get synchronized to Office 365. Unfortunately this means that all proxy addresses will be gone in Exchange Online!

It turns out this is not a configuration error, but a bug with this release... Microsoft statement is that “currently Proxyaddresses will not work with AADSYNC, and will be addressed in the next release”.

As such, you have two alternatives:
   1. Update to the latest version of AADSync, v1.0.0470.1023 (obviously recommended!);
   2. If, for some reason, you don’t want to upgrade, edit the existing rule to sync the proxyaddresses attribute.

Monday, November 3, 2014

Permissions lost after moving mailbox from Exchange 2003 to Exchange Online in hybrid environment

Consider the following scenario:
  • Your on-premises Exchange organization includes mailboxes that are hosted in Exchange 2003;
  • Your on-premises Exchange organization is set up for a hybrid deployment together with Exchange Online;
  • You move users or shared mailboxes from on-premises Exchange to Exchange Online.
After you move these mailboxes, you notice that the original mailbox permissions are not retained.

You might also notice that when you run the Add-MailboxPermission cmdlet in Exchange Online, you receive an error stating:
The ACL for the object “CN=user,CN=Users,DC=letsexchange,DC=com" is not in canonical order (Deny/Allow/Inherited) and will be ignored.

This is because Exchange 2003 uses a mailbox security descriptor system that is no longer used by Exchange Online. Because of this, when an Exchange 2003 mailbox is moved to Exchange Online, the original mailbox security descriptors are ignored and permissions are not kept.

To resolve this issue, run the FixMailboxSD command-line tool to correct the security descriptions on the on-premises Exchange 2003-based servers.

This is a small utility to fix mailbox security descriptors in Microsoft Exchange that have become non-canonical. It must be run on a machine with Exchange System Manager, as it relies on the interfaces exposed by CDOEXM, but it will work against mailboxes on 2003 or 2007 (not 2010 or 2013).

The tool uses CDOEXM from C# to read the MailboxRights object from the IExchangeMailbox interface. It then iterates through the DACL and puts all the ACEs in canonical order, and saves the changes.

The syntax of the tool is very straightforward:
FixMailboxSD “DN of mailbox”

For example:
FixMailboxSD “CN=nuno,CN=Users,DC=letsexchange,DC=com”

The tool will display a summary view of the current DiscretionaryAcl, and then show a summary view of the DACL after it has reordered it. It will then save the changes and return to a command prompt.

Wednesday, October 22, 2014

Exchange 2013 default OWA apps do not work

When you install Exchange Server 2013 on a Window Server 2012 R2-based computer, default apps (such as Bing map and Action item) in Outlook Web App do not work depending on the version of Exchange installed.
This issue occurs because the logic to load app resources is broken in Windows Server 2012 R2. To resolve this issue, install Cumulative Update 5 for Exchange Server 2013.

msExchangeRecipientTypeDetails Active Directory Values

A while back, while performing a migration to Office 365, I had to convert a Distribution Group into a Room List. However, due to the nature of the migration, I didn’t have access to an on-premises Exchange to use the Shell and convert it, so I had to resort to using ADSIedit. So how do we do this using ADSIedit?
There is a reference field that specifies what a recipient type is, as far as on-premises AD/Exchange is concerned, Recipient Type Details = msExchRecipientTypeDetails.
As many other AD attributes, these are represented by an Integer value in AD. Here are all the possible values for Recipient Type Details:

Object Type


Value Name

User Mailbox



Linked Mailbox



Shared Mailbox



Legacy Mailbox



Room Mailbox



Equipment Mailbox



Mail Contact



Mail User



Mail-Enabled Universal Distribution Group



Mail-Enabled Non-Universal Distribution Group



Mail-Enabled Universal Security Group



Dynamic Distribution Group



Public Folder


Public Folder

System Attendant Mailbox



System Mailbox



Cross-Forest Mail Contact









Universal Distribution Group



Universal Security Group



Non-Universal Group



Disabled User



Microsoft Exchange



Arbitration Mailbox



Mailbox Plan



Linked User



Room List



Discovery Mailbox



Role Group



Remote Mailbox



Team Mailbox


As such, all I had to do was locate the Distribution Group in AD, update its msExchRecipientTypeDetails attribute to 268435456 and wait for DirSync to replicate the change.