8000 Allow and control sending UTF-8 emails in the default email sender impl by rmartinc · Pull Request #41715 · keycloak/keycloak · GitHub
[go: up one dir, main page]

Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 10 additions & 0 deletions docs/documentation/server_admin/topics/realms/email.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -69,6 +69,16 @@ Auth Token Client Secret::
Only needed when *Authentication Type* 'token' is selected.
Supply the *Auth Client Secret* that authenticates the client to fetch a token from the *Auth Token URL*. The value of the *Auth Client Secret* field can refer a value from an external <<_vault-administration,vault>>.

Allow UTF-8::
Enable to UTF-8-encode email address when sending them to the server. This should only be enabled if the mail server supports UTF-8 via the SMTPUTF8 extension. If disabled, domain names containing non-ASCII characters will be encoded using punycode, and addresses containing non-ASCII characters in the local part of the address will return an error.
+
If you do not enable this option, take additional measures to prevent non-ASCII characters in users' email addresses:
+
--
. Verifying that no email addresses of existing users have non-ASCII characters in the local part of the email address.
. Updating the validation of email addresses to prevent non-ASCII characters in the local part of the email address, for example, by adding a regex pattern validation in the user profile for the email address field similar to `\p&#123;ASCII&#125;*@.*` with an error message similar to `Local part of the address must contain only ASCII characters`.
--

ifeval::[{project_community}==true]

== XOAUTH2 email configuration with third-party vendors
Expand Down
16 changes: 16 additions & 0 deletions docs/documentation/upgrading/topics/changes/changes-26_4_0.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -119,6 +119,22 @@ The input fields in the login theme for OTP and recovery codes and have been op

* The input mode is now `numeric`, which will ease the input on mobile devices.
* The auto-complete is set to `one-time-code` to avoid interference with password managers.
=== UTF-8 management in the email sender

Since this release, {project_name} adds a new option `allowutf8` for the realm SMTP configuration (*Allow UTF-8* field inside the *Email* tab in the *Realm settings* section of the Admin Console).
For more information about email configuration, see the link:{adminguide_link}#_email[Configuring email for a realm] chapter in the {adminguide_name}.

Enabling the option encodes email addresses in UTF-8 when sending them, but it depends on the SMTP server to also supports UTF-8 via the SMTPUTF8 extension.
If *Allow UTF-8* is disabled, {project_name} will encode the domain part of the email address (second part after `@`) using punycode if non-ASCII characters are used, and will reject email addresses that use non-ASCII characters in the local part.

If you have an SMTP server configured for your realm, perform the following migration after the upgrade:

* If your SMTP server supports SMTPUTF8:
. Enable the *Allow UTF-8* option.
* If your SMTP server does not support SMTPUTF8:
. Keep the *Allow UTF-8* option disabled.
. Verify that no email addresses of users have non-ASCII characters in the local part of the email address.
. Update the validation of email addresses to prevent allow non-ASCII characters in the local part of the email address, for example, by adding a regex pattern validation in the user profile for the email address field similar to `\p&#123;ASCII&#125;*@.*` with an error message similar to `Local part of the address must contain only ASCII characters`.

// ------------------------ Deprecated features ------------------------ //
== Deprecated features
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -40,6 +40,7 @@ displayName=Display name
applyToResourceTypeHelp=Specifies if this permission should be applied to all resources with a given type. In this case, this permission will be evaluated for all instances of a given resource type.
cibaIntervalHelp=The minimum amount of time in seconds that the CD (Consumption Device) must wait between polling requests to the token endpoint. If set to 0, the CD must use 5 as the default value according to the CIBA specification.
envelopeFrom=Envelope from
allowutf8=Allow UTF-8
eventTypes.UPDATE_TOTP.name=Update totp
updateCibaError=Could not update CIBA policy\: {{error}}
policyUrl=Policy URL
Expand Down Expand Up @@ -2733,6 +2734,7 @@ mappingCreatedError=Could not create mapping\: '{{error}}'
deleteClientPolicyProfileConfirmTitle=Delete profile?
passwordPoliciesHelp.forceExpiredPasswordChange=The number of days the password is valid before a new password is required.
envelopeFromHelp=An email address used for bounces (optional).
allowutf8Help=Enable to allow UTF-8 characters in the local part of the email address. This should only be enabled if the mail server supports UTF-8 via the SMTPUTF8 extension. If disabled, domain names containing UTF-8 characters will be encoded using punycode, and addresses containing UTF-8 characters in the local part of the address will return an error.
passwordPoliciesHelp.upperCase=The number of uppercase letters required in the password string.
policyDeletedError=Could not remove the resource {{error}}
key=Key
Expand Down
10 changes: 10 additions & 0 deletions js/apps/admin-ui/src/realm-settings/EmailTab.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -302,6 +302,16 @@ export const RealmSettingsEmailTab = ({
)}
</>
)}
<SwitchControl
name="smtpServer.allowutf8"
label={t("allowutf8")}
labelIcon={t("allowutf8Help")}
data-testid="smtpServer.allowutf8"
defaultValue=""
labelOn={t("enabled")}
labelOff={t("disabled")}
stringify
/>
<Controller
name="smtpServer.debug"
control={control}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -138,7 +138,7 @@ public void testWorkingRealmImportWithReplacement() {
k8sclient.getKubernetesSerialization().registerKubernetesResource(KeycloakRealmImport.class);
K8sUtils.set(k8sclient, getClass().getResourceAsStream("/example-realm.yaml"), obj -> {
KeycloakRealmImport realmImport = (KeycloakRealmImport) obj;
realmImport.getSpec().getRealm().setSmtpServer(Map.of("port", "${MY_SMTP_PORT}", "host", "${MY_SMTP_SERVER}"));
realmImport.getSpec().getRealm().setSmtpServer(Map.of("port", "${MY_SMTP_PORT}", "host", "${MY_SMTP_SERVER}", "from", "admin@keycloak.org"));
realmImport.getSpec().setPlaceholders(Map.of("MY_SMTP_PORT", new Placeholder(new SecretKeySelectorBuilder().withName("keycloak-smtp-secret").withKey("SMTP_PORT").build()),
"MY_SMTP_SERVER", new Placeholder(new SecretKeySelectorBuilder().withName("keycloak-smtp-secret").withKey("SMTP_SERVER").build())));
return realmImport;
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -32,4 +32,11 @@ default void send(Map<String, String> config, UserModel user, String subject, St
}

void send(Map<String, String> config, String address, String subject, String textBody, String htmlBody) throws EmailException;

/**
* Validates configuration for the SMTP sender.
* @param config The configuration to test
* @throws EmailException If some error is found
*/
void validate(Map<String, String> config) throws EmailException;
}
Original file line number Diff line number Diff line change
Expand Up @@ -17,13 +17,14 @@

package org.keycloak.email;

import jakarta.mail.internet.MimeUtility;
import org.jboss.logging.Logger;
import org.keycloak.common.enums.HostnameVerificationPolicy;
import org.keycloak.models.KeycloakSession;
import org.keycloak.models.UserModel;
import org.keycloak.services.ServicesLogger;
import org.keycloak.truststore.JSSETruststoreConfigurator;
import org.keycloak.utils.EmailValidationUtil;
import org.keycloak.utils.SMTPUtil;

import jakarta.mail.Address;
import jakarta.mail.MessagingException;
Expand All @@ -36,6 +37,7 @@
import jakarta.mail.internet.MimeBodyPart;
import jakarta.mail.internet.MimeMultipart;
import jakarta.mail.internet.MimeMessage;
import jakarta.mail.internet.MimeUtility;
import javax.net.ssl.SSLContext;
import javax.net.ssl.SSLSocketFactory;
import java.io.UnsupportedEncodingException;
Expand Down Expand Up @@ -74,24 +76,34 @@ public void send(Map<String, String> config, UserModel user, String subject, Str

@Override
public void send(Map<String, String> config, String address, String subject, String textBody, String htmlBody) throws EmailException {
Session session = Session.getInstance(buildEmailProperties(config));
final boolean allowutf8 = isAllowUTF8(config);
final String convertedAddress = checkUserAddress(address, allowutf8);
final String from = checkFromAddress(config.get("from"), allowutf8);

Message message = buildMessage(session, address, subject, config, buildMultipartBody(textBody, htmlBody));
Session session = Session.getInstance(buildEmailProperties(config, from));

Message message = buildMessage(session, convertedAddress, from, subject, config, buildMultipartBody(textBody, htmlBody));

try(Transport transport = session.getTransport("smtp")) {

EmailAuthenticator selectedAuthenticator = selectAuthenticatorBasedOnConfig(config);
selectedAuthenticator.connect(this.session, config, transport);

transport.sendMessage(message, new InternetAddress[]{new InternetAddress(address)});
transport.sendMessage(message, new InternetAddress[]{new InternetAddress(convertedAddress)});

} catch (Exception e) {
ServicesLogger.LOGGER.failedToSendEmail(e);
throw new EmailException("Error when attempting to send the email to the server. More information is available in the server log.", e);
}
}

private Properties buildEmailProperties(Map<String, String> config) throws EmailException {
@Override
public void validate(Map<String, String> config) throws EmailException {
// just static configuration checking here, not really testing email
checkFromAddress(config.get("from"), isAllowUTF8(config));
}

private Properties buildEmailProperties(Map<String, String> config, String from) throws EmailException {
Properties props = new Properties();

if (config.containsKey("host")) {
Expand Down Expand Up @@ -137,9 +149,9 @@ private Properties buildEmailProperties(Map<String, String> config) throws Email
props.setProperty("mail.smtp.from", envelopeFrom);
}

String from = config.get("from");
if (from == null) {
throw new EmailException("No sender address configured in the realm settings for emails");
final boolean allowutf8 = isAllowUTF8(config);
if (allowutf8) {
props.setProperty("mail.mime.allowutf8", "true");
}

// Specify 'mail.from' as InternetAddress.getLocalAddress() would otherwise do a InetAddress.getCanonicalHostName
Expand All @@ -150,12 +162,8 @@ private Properties buildEmailProperties(Map<String, String> config) throws Email
return props;
}

private Message buildMessage(Session session, String address, String subject, Map<String, String> config, Multipart multipart) throws EmailException {
private Message buildMessage(Session session, String address, String from, String subject, Map<String, String> config, Multipart multipart) throws EmailException {

String from = config.get("from");
if (from == null) {
throw new EmailException("No sender address configured in the realm settings for emails");
}
String fromDisplayName = config.get("fromDisplayName");
String replyTo = config.get("replyTo");
String replyToDisplayName = config.get("replyToDisplayName");
Expand Down Expand Up @@ -224,6 +232,10 @@ private static boolean isSslConfigured(Map<String, String> config) {
return "true".equals(config.get("ssl"));
}

private static boolean isAllowUTF8(Map<String, String> config) {
return "true".equals(config.get("allowutf8"));
}

private static boolean isDebugEnabled(Map<String, String> config) {
return "true".equals(config.get("debug"));
}
Expand All @@ -236,6 +248,42 @@ private boolean isAuthTypeTokenConfigured(Map<String, String> config) {
return "token".equals(config.get("authType"));
}

private static String checkUserAddress(String address, boolean allowutf8) throws EmailException {
final String convertedAddress = convertEmail(address, allowutf8);
if (convertedAddress == null) {
throw new EmailException(String.format("Invalid address '%s'. If the address contains UTF-8 characters in the local part please ensure the SMTP server supports the SMTPUTF8 extension and enable 'Allow UTF-8' in the email realm configuration.", address));
}
return convertedAddress;
}

private static String checkFromAddress(String from, boolean allowutf8) throws EmailException {
final String covertedFrom = convertEmail(from, allowutf8);
if (from == null) {
throw new EmailException(String.format("Invalid sender address '%s'. If the address contains UTF-8 characters in the local part please ensure the SMTP server supports the SMTPUTF8 extension and enable 'Allow UTF-8' in the email realm configuration.",
from));
}
return covertedFrom;
}

private static String convertEmail(String email, boolean allowutf8) throws EmailException {
if (!EmailValidationUtil.isValidEmail(email)) {
return null;
}

if (allowutf8) {
// if allowutf8 the extension will manage both parts
return email;
}

// if no allowutf8, do the IDN conversion over the domain part
final String convertedEmail = SMTPUtil.convertIDNEmailAddress(email);
if (convertedEmail == null || !convertedEmail.chars().allMatch(c -> c < 128)) {
// now if there are non-ascii characters, we should send an error
return null;
}

return convertedEmail;
}

protected InternetAddress toInternetAddress(String email, String displayName) throws UnsupportedEncodingException, AddressException, EmailException {
if (email == null || "".equals(email.trim())) {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,7 @@
*/
package org.keycloak.services.managers;

import jakarta.ws.rs.BadRequestException;
import jakarta.ws.rs.ClientErrorException;
import jakarta.ws.rs.core.Response;
import org.keycloak.Config;
Expand Down Expand Up @@ -70,7 +71,9 @@
import java.util.HashSet;
import java.util.List;
import java.util.Optional;
import org.keycloak.email.EmailException;
import org.keycloak.utils.ReservedCharValidator;
import org.keycloak.utils.SMTPUtil;
import org.keycloak.utils.StringUtil;

/**
Expand Down Expand Up @@ -566,6 +569,7 @@ public RealmModel importRealm(RealmRepresentation rep, Runnable userImport) {
ReservedCharValidator.validate(rep.getRealm());
ReservedCharValidator.validateLocales(rep.getSupportedLocales());
ReservedCharValidator.validateSecurityHeaders(rep.getBrowserSecurityHeaders());
SMTPUtil.checkSMTPConfiguration(session, rep.getSmtpServer());
realm.setName(rep.getRealm());

// setup defaults
Expand Down Expand Up @@ -679,6 +683,8 @@ protected void rollbackImpl() {
session.clientPolicy().updateRealmModelFromRepresentation(realm, rep);

fireRealmPostCreate(realm);
} catch (EmailException e) {
throw new BadRequestException(e.getMessage());
} finally {
session.getContext().setRealm(currentRealm);
}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -56,6 +56,7 @@
import org.keycloak.common.VerificationException;
import org.keycloak.common.util.PemUtils;
import org.keycloak.email.EmailAuthenticator;
import org.keycloak.email.EmailException;
import org.keycloak.email.EmailTemplateProvider;
import org.keycloak.events.EventQuery;
import org.keycloak.events.EventStoreProvider;
Expand Down Expand Up @@ -111,6 +112,7 @@
import org.keycloak.utils.GroupUtils;
import org.keycloak.utils.ProfileHelper;
import org.keycloak.utils.ReservedCharValidator;
import org.keycloak.utils.SMTPUtil;

import java.io.InputStream;
import java.security.cert.X509Certificate;
Expand Down Expand Up @@ -474,6 +476,13 @@ public Response updateRealm(final RealmRepresentation rep) {
throw ErrorResponse.error(e.getMessage(), Status.BAD_REQUEST);
}

try {
SMTPUtil.checkSMTPConfiguration(session, rep.getSmtpServer());
} catch (EmailException e) {
logger.error(e.getMessage(), e);
throw ErrorResponse.error(e.getMessage(), Status.BAD_REQUEST);
}

try {
if (!Constants.GENERATE.equals(rep.getPublicKey()) && (rep.getPrivateKey() != null && rep.getPublicKey() != null)) {
try {
Expand Down
71 changes: 71 additions & 0 deletions services/src/main/java/org/keycloak/utils/SMTPUtil.java
Original file line number Diff line number Diff line change
@@ -0,0 +1,71 @@
/*
* Copyright 2025 Red Hat, Inc. and/or its affiliates
* and other contributors as indicated by the @author tags.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/

package org.keycloak.utils;

import java.net.IDN;
import java.util.Map;
import org.keycloak.email.EmailException;
import org.keycloak.email.EmailSenderProvider;
import org.keycloak.models.KeycloakSession;

/**
* SMTP utility methods.
*
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Probably a valid Javadoc would be useful.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done!

* @author rmartinc
*/
public class SMTPUtil {

private SMTPUtil() {
// static helper class
}

/**
* Validates the configuration using the email sender provider.
*
* @param session The keycloak session to use
* @param config The configuration to validate
* @throws EmailException If some error is found in the configuration
*/
public static void checkSMTPConfiguration(KeycloakSession session, Map<String, String> config) throws EmailException {
if (config == null || config.isEmpty()) {
return;
}

final EmailSenderProvider sender = session.getProvider(EmailSenderProvider.class);
sender.validate(config);
}

/**
* Converts an email address to its ASCII representation using punycode
* (IDN.toASCII) for the domain part. The local part is not modified.
*
* @param email The email to convert
* @return The converted email or null (if IDN.toASCII throws an exception)
*/
public static String convertIDNEmailAddress(String email) {
final int idx = email == null ? -1 : email.indexOf('@');
if (idx < 0) {
return email;
}
try {
return email.substring(0, idx) + '@' + IDN.toASCII(email.substring(idx + 1));
} catch (IllegalArgumentException e) {
return null;
}
}
}
Loading
0