Microsoft 365 security checks

Keel runs the open-source Maester test suite against your Microsoft 365 tenant about once a day. It runs through Keel’s own read-only app, so your tenant gives Keel no password or secret.

Open it from Integrations. Its row has View findings and Settings, and its page has a tab for each.

Connect your tenant

Only an owner or admin of your Keel workspace can connect it.

1. Open the Settings tab and click Connect with Microsoft.
2. A Global Administrator of your tenant approves the Keel Maester app on Microsoft’s consent screen.
3. Assign the Global Reader role to the app, as described below.
4. To include Exchange Online and Security & Compliance, switch them on under Services checked, as described below.

The first run starts within about six hours of connecting.

Assign the Global Reader role

The Microsoft Graph checks run without it. The Exchange Online and Security & Compliance checks cannot run until the app holds Global Reader.

1. Sign in to the Microsoft Entra admin center.
2. Open Roles & admins and select Global Reader.
3. Click Add assignments and pick the Keel Maester app.

Choose the services checked

On the Settings tab, Services checked lists what each run covers. Microsoft Entra and Microsoft Graph are always on. Exchange Online and Defender for Office 365, and Security & Compliance, are off until you switch them on. Click Save services. The change applies from the next daily run.

Three SharePoint external-sharing tests are read through Microsoft Graph with read-only access: whether external sharing is limited, whether it is restricted to an allowed list of domains, and whether guests can reshare files and folders shared with them. Maester’s other SharePoint tests need a permission that can change settings, so Keel does not run them.

The Settings tab also shows when the last run finished, whether it completed, and the Maester version it used.

Read the findings

The Findings tab is a table of every test from the latest run. Sort it by any column, and filter it by status, severity and source. Click a test’s title to open its own page, with Maester’s description of the test, what it found, and a link to more guidance. That page also links to the previous and next finding with the same status.

Disconnect

Click Disconnect on the Settings tab. Keel stops running checks against your tenant. To remove Keel’s access on the Microsoft side as well, delete the Keel Maester app from Enterprise applications in the Microsoft Entra admin center.