IMAP mailboxes

What IMAP covers

Any mailbox that speaks IMAP and SMTP can be connected with a password. That includes Yahoo, AOL, iCloud, Fastmail, GMX, WEB.DE, Yandex and a custom domain at a hosting company. Gmail and Zoho Mail still connect through their own sign-in (see Mailboxes), and Outlook or Microsoft 365 is coming.

Campbooks reads the mail, makes members' changes at the server (read, starred, archive, trash, spam and streams), and sends through the mailbox's own SMTP server. A copy of each sent message goes into the mailbox's Sent folder.

Turning it on

IMAP is on by default. It needs only the instance secret key, which seals each mailbox password. See The instance secret key.

• With the key set, Settings, then Mailboxes, then Add a mailbox offers IMAP for any address that is not Gmail, Zoho or Outlook.

• Without the key, the server still starts. It logs a warning and does not offer IMAP.

• To turn IMAP off, set this in .env and run docker compose up -d:

  CAMPBOOKS_CONNECTORS__IMAP__ENABLED=false

  Setting it to true by name while the key is missing stops the server, with an error that names the key.

There is no client to register and nothing to set up at the provider. Each member makes an app password at their provider; see Connect your mailbox for the steps at each provider.

How a member connects one

1. In Settings, then Mailboxes, the member chooses Add a mailbox (or presses Shift C) and types the address.
2. Campbooks finds how that mailbox connects:
   • Gmail and Zoho addresses go to the provider's own sign-in, without a password.
   • Outlook and Microsoft 365 addresses get a note that they are coming soon.
   • Every other address gets the app-password step. Known providers show their own steps and a link to their help page.
3. The member pastes the app password and chooses Connect. Before keeping anything, Campbooks signs in to the IMAP server and authenticates to the SMTP server.
4. An Advanced section shows the username and both servers. It is closed and filled in when the servers were found, and open when they were not.

How the servers are found

Campbooks looks in these places, in order:

1. A copy of Thunderbird's provider database, bundled with Campbooks.
2. The domain's own autoconfig document.
3. The domain's SRV records.
4. Its MX host.

The address is never sent to a third party to look it up.

Encryption only

Campbooks connects over TLS or STARTTLS only, and checks the certificate against the server's name. There is no plain-text option. A server that offers neither is refused before the password is sent.

The password

A mailbox password is worth more than an OAuth grant. It has no scope, and Campbooks cannot revoke it at the provider. Campbooks handles it like this:

• It is kept encrypted with the instance secret key and bound to its connection.
• It is never shown again and never logged.
• It is deleted when the mailbox is disconnected.

Ask members to use app passwords. They can revoke an app password at their provider without changing their main password.

The address check

A mailbox whose username differs from its address cannot send until the address is checked. When the username is the address itself, the sign-in already proves it.

1. Campbooks sends one plain-text message, from and to the address, through the mailbox's own SMTP server. Its Message-ID carries a one-time code.
2. Within 15 minutes, a sync of the mailbox finds that message in its Inbox.
3. The address is then marked verified, and the message is removed by its UID. Where the server cannot do that safely, it is moved to Trash, or left where it is.

The member types nothing. Settings, then Mailboxes, shows "Checking your address…", then Verified. If the message is not found in time, the mailbox reads mail but sending stays off, and the row offers Try again. A mailbox gets at most 5 checks a day.

The check has a limit. The member holds the mailbox password. Anyone who can see the code (because they run the server, or because the server files sent mail where they can read it) can pass the check. It stops a shared server that lets any sign-in send as any address from making someone else's address sendable through Campbooks. It does not stop the operator of the mail server.

Network rules

By default, mail servers are reached only at public addresses and only on ports 993, 143, 465 and 587. If your mail server is on your own network, or uses other ports, set:

CAMPBOOKS_CONNECTORS__IMAP__ALLOW_PRIVATE_HOSTS=true
CAMPBOOKS_CONNECTORS__IMAP__ALLOW_ANY_PORT=true

ALLOW_PRIVATE_HOSTS allows private, loopback and CGNAT addresses. Link-local and cloud metadata addresses are never allowed. Leave both off on an instance shared with people you do not know.

Connect attempts are limited to 10 per member and 30 per workspace an hour. A mailbox login is limited to 5 failed sign-ins an hour across the instance, so the screen cannot be used to guess passwords.

How it syncs

Campbooks polls IMAP mailboxes on the same schedule as the others:

• Within a minute after a sync that found changes.
• Backing off to CAMPBOOKS_CONNECTORS__POLL__MAX_SECONDS while the mailbox is quiet.

It reads every folder (up to 200), and every 6 hours it sweeps the flags of each mailbox's window. A new mailbox reads back 90 days.

A stream with no folder of its own gets a Campbooks/<Stream> folder, up to 10 per mailbox. Campbooks never deletes or renames a folder, and never empties one.

Push with IDLE (optional)

IDLE lets the server tell Campbooks about new mail at once, instead of waiting for the next poll. It is off by default. To turn it on:

CAMPBOOKS_CONNECTORS__IMAP__IDLE_ENABLED=true

While IDLE holds a mailbox, Campbooks still polls it as a safety net every CAMPBOOKS_CONNECTORS__IMAP__IDLE_POLL_SECONDS (900 by default). That value must sit between CAMPBOOKS_CONNECTORS__POLL__MIN_SECONDS and CAMPBOOKS_CONNECTORS__POLL__MAX_SECONDS; otherwise the server refuses to start with IDLE on. If you lowered POLL__MAX_SECONDS, lower IDLE_POLL_SECONDS with it.

Tuning

All of these are optional. Add them to .env and run docker compose up -d.

Variable	Default	What it does
CAMPBOOKS_CONNECTORS__IMAP__ENABLED	true	Offer IMAP mailboxes. Needs the instance secret key.
CAMPBOOKS_CONNECTORS__IMAP__ALLOW_PRIVATE_HOSTS	false	Allow mail servers on private, loopback and CGNAT addresses.
CAMPBOOKS_CONNECTORS__IMAP__ALLOW_ANY_PORT	false	Allow ports other than 993, 143, 465 and 587.
CAMPBOOKS_CONNECTORS__IMAP__CONNECT_TIMEOUT_SECONDS	10	Time to open a connection to a mail server.
CAMPBOOKS_CONNECTORS__IMAP__TIMEOUT_SECONDS	30	Time a connect check may take in all; while reading a mailbox, how long one command may wait.
CAMPBOOKS_CONNECTORS__IMAP__BACKFILL_DAYS	90	How far back a new mailbox reads, in days.
CAMPBOOKS_CONNECTORS__IMAP__MAX_MESSAGE_BYTES	52428800 (50 MiB)	Larger messages are skipped and recorded as oversized.
CAMPBOOKS_CONNECTORS__IMAP__RUN_BUDGET_BYTES	67108864 (64 MiB)	A sync run starts no download after this many bytes and continues in the next run.
CAMPBOOKS_CONNECTORS__IMAP__SWEEP_MINUTES	360	Minutes between sweeps of a mailbox's flags.
CAMPBOOKS_CONNECTORS__IMAP__SESSIONS_PER_HOST	4	IMAP sessions open at once to one server, across all processes.
CAMPBOOKS_CONNECTORS__IMAP__PROVIDER_SESSIONS_PER_HOST	200	The same, for a large provider's server (a short list kept in the code).
CAMPBOOKS_CONNECTORS__IMAP__WORKSPACE_SESSIONS_PER_HOST	2	Sessions one workspace may hold on one server.
CAMPBOOKS_CONNECTORS__IMAP__MAX_FOLDERS	200	The most folders read per mailbox.
CAMPBOOKS_CONNECTORS__IMAP__MAX_CREATED_FOLDERS	10	The most Campbooks/<Stream> folders made per mailbox.
CAMPBOOKS_CONNECTORS__IMAP__SEND_PER_HOUR	50	Messages one mailbox login sends an hour.
CAMPBOOKS_CONNECTORS__IMAP__SEND_RECIPIENTS_PER_MESSAGE	50	Recipients one message may have.
CAMPBOOKS_CONNECTORS__IMAP__SEND_RECIPIENTS_PER_DAY	500	Recipients one mailbox login sends to a day.
CAMPBOOKS_CONNECTORS__IMAP__SEND_WORKSPACE_PER_HOUR	200	Messages one workspace sends an hour.
CAMPBOOKS_CONNECTORS__IMAP__SEND_HOST_PER_HOUR	1000	Messages sent through one SMTP server an hour, by all workspaces. Large providers have no cap; elsewhere one workspace takes at most half.
CAMPBOOKS_CONNECTORS__IMAP__SEND_MAX_MESSAGE_BYTES	26214400 (25 MiB)	The largest message sent. At most 35 MiB.
CAMPBOOKS_CONNECTORS__IMAP__SEND_TIMEOUT_SECONDS	300	How long one submission may take. At most 600.
CAMPBOOKS_CONNECTORS__IMAP__IDLE_ENABLED	false	Push with IDLE.
CAMPBOOKS_CONNECTORS__IMAP__IDLE_SESSIONS	50	Mailboxes each worker holds in IDLE.
CAMPBOOKS_CONNECTORS__IMAP__IDLE_SESSIONS_PER_WORKSPACE	5	Of those, the most for one workspace.
CAMPBOOKS_CONNECTORS__IMAP__IDLE_DEBOUNCE_SECONDS	30	At most one sync per mailbox in this time when IDLE reports changes.
CAMPBOOKS_CONNECTORS__IMAP__IDLE_REFRESH_MINUTES	25	How often IDLE is renewed. At most 29.
CAMPBOOKS_CONNECTORS__IMAP__IDLE_QUIET_DAYS	14	A mailbox quiet this long goes back to polling.
CAMPBOOKS_CONNECTORS__IMAP__IDLE_POLL_SECONDS	900	The safety poll of a mailbox held in IDLE.
CAMPBOOKS_CONNECTORS__IMAP__WORKSPACE_IDLE_SESSIONS_PER_HOST	10	IDLE sessions one workspace may hold on one server.

Troubleshooting

What the member sees	Cause and fix
"The mail server refused the sign-in"	Wrong password, or the provider needs an app password and the usual password was used. Make an app password and paste that. At Yandex, a new app password can take a few hours to start working.
"Campbooks couldn't find this mailbox's servers"	The domain publishes no settings. Enter the servers under Advanced: the host's help pages list them.
"Campbooks couldn't reach the mail server"	The name or port is wrong, or a firewall blocks the server. Check the servers under Advanced.
"Campbooks can't reach that server: it's on a private network…"	The server is on a private address, or uses a port other than 993, 143, 465 or 587. On your own instance, set ALLOW_PRIVATE_HOSTS or ALLOW_ANY_PORT (see Network rules).
"The mail server's certificate isn't valid for its name"	The server name does not match its certificate. Use the name on the certificate, often the hosting company's server name rather than mail. plus your domain.
"That server doesn't offer an encrypted connection"	Use port 993 for IMAP and 465 for SMTP, with TLS.
"Too many failed sign-ins to this mailbox"	Five failed sign-ins in an hour. Wait an hour.
"We couldn't verify this address: sending stays off"	The check message did not reach the Inbox within 15 minutes. It may have landed in Spam. Choose Try again, which sends a new one; if it fails again, check the mailbox's spam filter.
GMX or WEB.DE refuses the sign-in	IMAP access is off in the mailbox's settings. Turn it on (POP3 and IMAP), then connect again. Both providers switch it off again after long periods without use.
IMAP is not offered at all	The instance secret key is not set, or CAMPBOOKS_CONNECTORS__IMAP__ENABLED=false. Check docker compose logs server for the warning.