The Greenhouse Recruiting platform suffered an email delivery issue that began early on March 15 and was resolved on March 18, 2019. This issue affected emails sent to Microsoft email addresses (for example, hotmail.com and outlook.com).
Microsoft’s email service provider flagged one of the Greenhouse Recruiting IP addresses as a potential spammer and blacklisted this IP address. Greenhouse Recruiting has a pool of IP addresses through which it can send emails. As a result, a portion of emails sent from Greenhouse Recruiting to a Microsoft mailbox failed to deliver. When emails fail to be delivered, Greenhouse users are notified via an automated email receipt.
Greenhouse engineers diagnosed the problem late Friday afternoon and began to mitigate the issue. Additional IP addresses were added to the Greenhouse Recruiting email pool. These new IP addresses were placed in a "warm-up mode" to prevent them from being marked as spam senders immediately.
After consulting with our mail provider, Mailgun, we decided not to remove the blacklisted IP address from the pool on Friday. We had reason to believe that the block on the IP address by Microsoft was temporary and would be removed without any action after 24 hours. However, the block did not appear to be removed by Monday morning, and some emails from Greenhouse Recruiting were still failing to be delivered. Greenhouse engineers responded by moving more traffic onto the newly-added IP addresses. After further consultation with Mailgun, we decided to move forward with removing the blacklisted IP address from the pool, which resolved the issue.
On March 19, 2019, Mailgun confirmed that Microsoft mitigated the IP address that caused the issue and removed the IP address from their block list. We are continuing to investigate why the IP was blacklisted, and Greenhouse engineers will be working with Mailgun to ensure that we are following best practices for email deliverability to help prevent issues like this moving forward. We’ll also create an action plan for how to handle this type of issue should Microsoft block another of our IP addresses in the future.