Proton Administration

Proton is a Domino server add-in task that is part of the Domino AppDev Pack. The purpose of this task is to execute remote requests from applications written to use the Node.js domino-db module. Applications can perform operations on documents in a server database.

This version of Proton has been verified to run with these Domino releases:

Domino 10.0.1 FP4
Domino 11

Installation

The Proton installation tar/zip file contains the following:

The Proton Domino server addin task and the required shared library (libiamclient.so/iam-client.dll)
OAuth configuration tool
OAuth Domino Server plugin (DSAPI)
Sample scripts for creating client and server SSL certificates and Domino keyring

Linux Installation

The Proton Domino addin task ships as compressed tar file named <domino>-proton-addin-<version>.tgz. To install the addin task on your Domino server do the following:

Switch to the Domino binary directory Note: Before Domino 11, the bin directory was in /opt/ibm/domino/
```
cd /opt/hcl/domino/notes/latest/linux
```
Verify that libnotes.so exists in this directory
```
ls -la libnotes.so
```
Locate the correct file that matches the Domino version of your server:
- Example: For a Domino 10.01 server running FP4, use 1001-proton-addin-<version>.tgz
- Example: For a Domino 11 server, use 1100-proton-addin-<version>.tgz

Expand the proton-addin.tgz file

sudo tar -xvf /tmp/<domino>proton-addin-<version>.tgz

Fixup file permissions and ownership
```
sudo sh -v ./setup_proton.sh
```

Windows Installation

The Proton Domino addin task ships as a compressed zip file named <domino>-proton-addin-<version>.zip. To install the addin task on your Domino server do the following:

switch to the Domino binary directory
```
cd c:\Program Files\HCL\Domino
```
verify that nnotes.dll exists in this directory
```
dir nnotes.dll
```
Locate the correct file that matches the Domino version of your server:
- Example: For a Domino 10.01 server running FP4, use 1001-proton-addin-<version>.zip
- Example: For a Domino 11 server, use 1100-proton-addin-<version>.zip
expand the proton-addin.zip file
1. Using Windows explorer, select the <domino>-proton-addin-<version>.zip file.
2. Right-click and select "Extract all".
3. Specify the Domino binary directory, c:\Program Files\HCL\Domino, as the destination and click "Extract".

Starting and stopping the Proton add-in task

Use the load Domino server console command to load the Proton task:

> load proton
PROTON> Build <version>
PROTON> Listening on 127.0.0.1, port 38770, INSECURE
PROTON> Note: Requested port was 0, Actual listen port is 38770
PROTON> Server initialized
PROTON> Server only allows Anonymous access.

Use the tell command to shutdown the Proton task:

> tell proton quit
PROTON> Server stopping
PROTON> Server exited

Automatically starting Proton at Domino Server startup

Domino has a notes.ini which specifies the addin tasks to automatically start at server startup. The notes.ini is ServerTasks and it contains all the task names to automatically load. Proton needs to be added to this notes.ini. You can edit the server's notes.ini directly with an editor if the server is not running, or use the Domino server console commands to add PROTON to the list. For example:

> show config servertasks
[001793:000008-0000003243423488] SERVERTASKS=Replica,Router,Update,AMgr,Adminp,Sched,CalConn,RnRMgr,LDAP
> set config SERVERTASKS=Replica,Router,Update,AMgr,Adminp,Sched,CalConn,RnRMgr,LDAP,PROTON

Proton Configuration

Listen server address and port

By default, Proton will select an unused port on the server and listen on that port. This behavior is useful for testing purposes only. For production environments the administrator should set the specific address and port for the Proton listener.

The notes.ini variables that control this behavior are:

PROTON_LISTEN_ADDRESS The address where Proton will listen for network connections. The default value is "127.0.0.1", which means that Proton will accept connections only from the local machine. Use "0.0.0.0" to bind to all ip addresses available on the machine.

PROTON_LISTEN_PORT The port where Proton will listen for network connections. The default is 0, which means that Proton will select an available port on the machine. This option is useful for testing purposes only. Specify a port that is available for use on the machine.

TLS/SSL enablement

By default, Proton will provide Anonymous access to databases on the server. Database access is also governed by the 'Maximum Internet name and password' specified on the database ACL.

By default, network traffic to/from Proton is transmitted unencrypted. To encrypt traffic specify the following notes.ini variables:

PROTON_SSL Set to 1 to enable SSL on the PROTON_LISTEN_PORT. The Proton server will accept either SSL connections or clear-text connections. It will not accept both at the same time.

PROTON_KEYFILE Set to the key file created by the Domino KYRTOOL tool. The keyfile must have a private key for the Proton add-in task. The file path is relative to the Domino server's data directory.

Client authentication

Proton authenticates client application requests based on the setting of the PROTON_AUTHENTICATION notes.ini setting. Valid options are:

client_cert: The client certificate is mapped to a Person document in the server's directory. Access to data is calculated based on this identity. Proton must be enabled for TLS/SSL for this option.

anonymous: All requests are made as the Anonymous user identity. This name does not need to appear in the directory, but it does need to exist in database ACL. This option is available with and with out TLS/SSL being enabled.

The default behavior when the setting does not exist is to provide Anonymous access to Domino databases.

Client certificate authentication

To require that applications provide a valid client certificate set the following notes.ini variable:

PROTON_AUTHENTICATION=client_cert

With this setting enabled there are some additional administrative and client requirements.

The client application must supply a valid client certificate when making domino-db requests to the Proton server on Domino. The common name in the client certificate must have a name that can found in the Domino directory. Proton performs a lookup in the Domino directory to find the person document.
The Domino administrator must create a Person document in the Domino directory and perform the Import Internet Certificates Action on the Person document. This is required because the client certificate is verified against the known certificate in the Domino directory.

example person with imported certificate

Tool for loading client certificate into the Domino directory

The Proton add-in task can also be used to import the client certificate into a person document. Use the load proton --importcert certfilename.crt command on the Domino server to have proton add the certificate for a functional id into the Domino directory. For example:

> load proton --importcert testapp1.crt
07/08/2019 12:33:13.67 PM PROTON_CMD> import_certificate: testapp1.crt
07/08/2019 12:33:13.68 PM PROTON_CMD> Certificate::init: subject: CN=TestApp1/OU=Applications/O=Develop
07/08/2019 12:33:13.74 PM PROTON_CMD> Updating note 0x1D96 for 'TestApp1'

By using this method to load the certificate into the directory you can verify that the client application using the certificate will be mapped to the correct functional id in the directory. The command requires only the file name that has the client certificate. Proton matches this certificate to a user in the Domino directory by the simple name in the certificate, and is the same lookup that Proton uses when the application sends a request via domino-db.

Sample Certificate Management Scripts - Linux

There are two sample scripts provided with proton-addin.tgz which show how to create a certificate authority, client and server keys and the keyring. Consult with your organization's security expert before using these scripts.

make_certs.sh: creates a certificate authority and a few sample certificates and keys using the openssl command line utility. Client applications using the domino-db module will need their own certificate and key when Proton is enabled for SSL connections.
make_keyring.sh: shows how to use the Domino KYRTool to create a keyring file that Proton can use for SSL connections. The script uses the sample server key and certificate by make_certs.sh.

You can find additional information about using the Domino KYRTool on this page.

Sample Certificate Management Scripts - Windows

Prerequisites

The following scripts require a Windows distribution of OpenSSL. You can find additional information about using OpenSSL on Windows on this page

Sample Scripts

There are two sample scripts provided with proton-addin.zip which show how to create a certificate authority, client and server keys and the keyring. Consult with your organization's security expert before using these scripts.

make_certs.cmd: creates a certificate authority, a sample server certificate (with private key) and a two sample application certificates (also with private keys) using the openssl command line utility. Run make_certs <cert directory> where <cert directory> is the destination for the generated certificates and keys.
make_keyring.cmd: shows how to use the Domino KYRTool to create a keyring file that Proton can use for SSL connections. The script uses the sample server key and certificate created by make_certs.sh. Two files are created: sample.kyr and sample.sth. Both can be renamed and copied to a convenient location for example: c:\datadir\proton.kyr and c:\datadir\proton.sth.

Summary of a reasonable set of Proton options for production use

PROTON_SSL=1
PROTON_LISTEN_PORT=3002
PROTON_LISTEN_ADDRESS=0.0.0.0
PROTON_AUTHENTICATION=client_cert
PROTON_KEYFILE=proton.kyr

Note: After changing any of the above configuration variables, it will be necessary to restart the Proton task.

Act as User

The following notes.ini variables control aspects of this feature for all applications using Proton. The Act-as-User feature can be used only when Client certificate authentication is enabled in Proton.

PROTON_ACTASUSER (boolean) allowed values are 0 or 1 indicating whether this feature is available to domino-db applications. When this value is '0' Act-as-User requests will fail with the FEATURE_NOT_ENABLED error. If this variable is missing, and Client Certificate authentication is enabled, Proton defaults to Act-as-User feature being enabled.
PROTON_IAMCLIENT_CONFIG_NAME (string) This is the name used with the oauthcfg tool to create the IAM introspection credentials. The introspection credentials are created by IAM when you register a resource provider. You then store these credentials in the Domino credential store using the oauthcfg tool under a name that you provide with the oauthcfg create command. Proton uses the value of this variable to find the credentials and URL that it will use to make introspection requests to IAM. See Introspection Credentials for more details. If this variable is not set, Proton will use the default value: proton.

Note: After changing any of the above configuration variables, it will be necessary to restart the Proton task.

Additional configuration variables

The following notes.ini variables are not required in most cases, but they may help you tune your Proton configuration:

PROTON_MAX_NOTE_COUNT controls the maximum number of documents Proton will process in a query found set. By default, Proton will operate on a maximum of 200 documents per request. The following server console command increases the maxiumum to 1000 documents:
```
set config PROTON_MAX_NOTE_COUNT=1000
```
Warning: If Proton executes many simultaneous queries with large found sets, the PROTON_MAX_NOTE_COUNT setting can affect overall performance. You should find the right balance based on the applications running in your enterprise.
PROTON_MAX_WRITE_ATTACHMENT_MB controls the maximum size of an attachment Proton will write to any database. By default, there is no limit. The following server console command sets a limit of 500 Mb:
```
set config PROTON_MAX_WRITE_ATTACHMENT_MB=500
```
PROTON_MAX_ATTACHMENT_CHUNK_KB controls the maximum chunk size Proton will use when a client is reading an attachment. By default, the maximum chunk size is 32 Kb. The following server console command sets the maximum chunk size to 48 Kb:
```
set config PROTON_MAX_ATTACHMENT_CHUNK_KB=48
```
You cannot increase the maximum chunk size beyond 64 Kb or decrease it below 8 Kb.
PROTON_MIN_ATTACHMENT_CHUNK_KB controls the minimum chunk size Proton will use when a client is reading an attachment. By default, the minumum chunk size is 4 Kb. The following server console command increases it to 8 Kb:
```
set config PROTON_MIN_ATTACHMENT_CHUNK_KB=8
```
You cannot increase the minimum chunk size beyond 8 Kb or decrease it below 1 Kb.

Act-as-User Database Configuration

Applications must be given permission to perform Act-as-User operations on behalf of end users using domino-db.

Each domino-db application should use a different user in the Domino directory, known as the application's functional id. The application's functional id must have the _ActAsUser role in the database to be able to make Act-as-User operations.

Furthermore, an application may or may not have access via the ACL to perform operations with it's own identity. The minimum access that an application may have to be able to make Act-as-User requests is Read public documents as shown below.

In this example, the application has Act-As-User rights, but is only able to read public documents with it's own identity.

Act-as-User OAuth Configuration

Proton relies on the IAM service to validate and decode the access tokens passed via the domino-db API. This operation is known as introspection and is defined by RFC 7662. The following configuration steps are necessary to allow proton to securely perform introspection calls to the IAM service:

Configure the Domino Credential Store.
Verify the hostname and port of the IAM introspection endpoint. This will be the value of the Issuer setting as described here
Create a resource provider on the IAM service.
1. Go to the IAM Management Dashboard, click Manage Resource Providers and create a "Resource Provider". After clicking "Create", enter descriptive values that will identify this resource provider as the proton resource provider. For example:
  1. Resource Provider Name: Proton.
  2. Resource Provider Description: Proton resource provider.
  3. Click Save. You are shown the Resource Provider ID and Resource Provider Secret. (You can come back to these if needed.)

Add the resource provider credentials to the Domino server credential store.
Linux:

Open a command prompt and then run the following commands:

Note: Before Domino 11, the bin directory was in /opt/ibm/domino/

cd /local/notesdata
/opt/hcl/domino/bin/tools/startup /opt/hcl/domino/notes/latest/linux/oauthcfg create proton <Resource Provider ID> <Resource Provider Secret> <https://<domino-iam-service host>/token/introspection>

Windows:

Open a command prompt and then run the following commands:

cd <notes.ini-location>
\<domino-program-directory>\oauthcfg create proton <Resource Provider ID> <Resource Provider Secret> <https://<domino-iam-service host>/token/introspection>

See Introspection Credentials for more details.

Add a certificate capable of verifying the IAM service certificate to the proton keyring file. There are three possibilities:
1. If a self-signed root CA (Certificate Authority) was used to create the IAM service certificate, then the root certificate associated with the CA needs to be added to the proton keyring.
  or
2. If an intermediate CA was used to create the IAM service certificate (this is likely the case when using a public CA), then the certificate associated with the root CA (the certificate at the top of the certificate chain) needs to be added to the proton keyring.
  or
3. Import the IAM service certificate into the proton keyring file.

Caches

Proton maintains the following cache:

Session Cache

Requests from the domino-db module are authenticated. This operation includes the following steps which can be expensive to perform on every request:

Directory lookup to identify the functional id in the Domino directory
Client certificate validation against the Domino directory
Building of the nameslist and group expansion for the functional id
Retrieval of the id file for the functional id when needed

To improve application peformance these operations will be performed once every 5 minutes for each functional id. When the maximum of 5000 functional ids are in the cache the least recently used sessions will be ejected from the cache to maintain this maximum limit.

Statistics

The Proton addin task maintains a set of Domino statistics which provide a view into operational aspects of this service. Access the Proton statistics using the Domino server console command: show stat proton.

For example:

> show stat proton
PROTON.Available = 1
PROTON.Available.ElapsedSecs = 407376
PROTON.Request.Errors = 3428193
PROTON.Request.Errors.OutOfMemory = 0
PROTON.Request.QueryDB = 189617
PROTON.Request.QueryDB.Errors = 68952
PROTON.Request.QueryDB.Explain = 25854
PROTON.Request.Total = 4197796
PROTON.Session.Cache1.Count = 0
PROTON.Session.Cache1.Entries.Bytes = 0
PROTON.Session.Cache1.Entries.Count = 0
PROTON.Session.Cache1.LockTimeMS = 34685
PROTON.Session.Cache1.LockWaitTimeMS = 8648
PROTON.Session.Cache1.Lookup.Expired = 212
PROTON.Session.Cache1.Lookup.Hit = 1102824
PROTON.Session.Cache1.Lookup.Miss = 15
PROTON.Session.Cache1.Prune.Ejected = 0
PROTON.Session.Cache1.Prune.Expired = 6
PROTON.Session.Cache1.Save = 218
PROTON.Session.Cache1.Save.Miss = 9
PROTON.Session.IdCache.Errors = 0
PROTON.Session.IdCache.Miss = 47
PROTON.Session.Init.Errors = 3103365
PROTON.Session.Lookup.Certificate = 227
PROTON.Session.Lookup.Certificate.Errors = 0
PROTON.Session.Lookup.IdVault = 290
PROTON.Session.Lookup.IdVault.ConcurrentMax = 3
PROTON.Session.Lookup.IdVault.Errors = 25
28 statistics found

Notes about the Proton statistics:

Session.Cache1.* statistics refer to the certificate based authentication cache operations.
Session.IdCache.* statistics refer to the use of the Id Vault cache when an application makes use of encryption operations.
Session.Lookup.* statistics refer to request authentication operations.
Request.* statistics refer to application request operations.

Next Steps

Now that Proton is set up. You may want to create the DQL Design Catalog Database.

Troubleshooting

Verify that the Proton server task is ready to accept network connections with SSL enabled

OpenSSL is a tool that can be used to verify that the Proton is available to be used with SSL and to review the configuration settings.

The following commands work equally well on recent Windows (10+) and Linux machines.

OpenSSL provides a lot of very detailed information. Reading the output takes some practice. These are some the key messages to look for in the OpenSSL output:

ssl handshake failure - Indicates the SSL negotiation between the client and the server failed.
connect:errno=9 - Indicates that the connection to the Proton server failed - errno could provide some additional clues about the reason.

Proton is not accepting network connections

Possible reasons for this are:

Proton task is not running, or it's not listening on a different port
DNS does not resolve the hostname to the desired server
Network connectivity between client and server machines is blocked (e.g. a firewall is in place)

> openssl s_client -connect domino-server1.example.com:3002
18548:error:0200274D:system library:connect:reason(1869):crypto/bio/b_sock2.c:108:
18548:error:2008A067:BIO routines:BIO_connect:connect error:crypto/bio/b_sock2.c:109:
connect:errno=0

Verify/Review Proton's SSL configuration

When the client is able to connect to the Proton server task the OpenSSL command above will display important details about the SSL configuration. The CONNECTED message indicates that there was a successful connection made between the client machine and Proton.

$ openssl s_client -connect domino-server1.example.com:3002
CONNECTED(00000003)

Output from OpenSSL continues with details about the Proton certificate:

depth=1 O = Test, CN = TestCA
verify error:num=19:self signed certificate in certificate chain
140189510182800:error:140790E5:SSL routines:ssl23_write:ssl handshake failure:s23_lib.c:177:
---
Certificate chain
 0 s:/O=Test/CN=proton-server
   i:/O=Test/CN=TestCA
 1 s:/O=Test/CN=TestCA
   i:/O=Test/CN=TestCA
---
Server certificate
-----BEGIN CERTIFICATE-----
... (Large number of HEX digits removed) ...
-----END CERTIFICATE-----
subject=/O=Test/CN=proton-server
issuer=/O=Test/CN=TestCA
---

Followed by the list of Certificate Authorities that Proton can verify when the client presents a client certificate. Client certificates are required for all Proton clients.

Acceptable client certificate CA names
/O=Test/CN=TestCA
/O=VeriSign Trust Network/OU=VeriSign, Inc./OU=VeriSign International Server CA - Class 3/OU=www.verisign.com/CPS Incorp.by Ref. LIABILITY LTD.(c)97 VeriSign
... (Additional Certificate Authorities removed) ...
/C=IE/O=Baltimore/OU=CyberTrust/CN=Baltimore CyberTrust Root
/C=IE/O=Baltimore/OU=CyberTrust/CN=Baltimore CyberTrust Mobile Root

Followed by additional details about the SSL connection:

Client Certificate Types: RSA sign, ECDSA sign
Requested Signature Algorithms: ECDSA+SHA256:0x04+0x08:RSA+SHA256:ECDSA+SHA384:0x05+0x08:RSA+SHA384:0x06+0x08:RSA+SHA512:RSA+SHA1
Shared Requested Signature Algorithms: ECDSA+SHA256:RSA+SHA256:ECDSA+SHA384:RSA+SHA384:RSA+SHA512:RSA+SHA1
Peer signing digest: SHA256
Server Temp Key: ECDH, P-256, 256 bits
---
SSL handshake has read 7029 bytes and written 138 bytes
---
New, TLSv1/SSLv3, Cipher is ECDHE-RSA-AES256-GCM-SHA384
Server public key is 4096 bit
Secure Renegotiation IS supported
Compression: NONE
Expansion: NONE
No ALPN negotiated
SSL-Session:
    Protocol  : TLSv1.2
    Cipher    : ECDHE-RSA-AES256-GCM-SHA384
    Session-ID:
    Session-ID-ctx:
    Master-Key: 114FE6BCC3389BA58E74040B0A19BCAD1A1337CF2D859E0FD8038BD443A00B820A1B90B7892EFFB873EE52C44293EBE6
    Key-Arg   : None
    Krb5 Principal: None
    PSK identity: None
    PSK identity hint: None
    Start Time: 1541078806
    Timeout   : 300 (sec)
    Verify return code: 19 (self signed certificate in certificate chain)
---

Testing the connection with the client certificate

Clients connecting to the Proton server must present a client certificate to be able to send and receive gRPC messages. To verify that the client is using a certificate that can be verified by the Proton server use this OpenSSL command:

$ openssl s_client -connect domino-server1.example.com:3002 -cert app2.crt -key app2.key

A succesful SSL connection with the above command proves that the Proton server is able to trust the client certificate. It does not prove that the application will be able to perform domino-db operations with that client certificate. When using the OpenSSL command above, Domino has not performed authentication and authorization against the Domino directory. The client must perform an actual request for that to happen. Use the domino-db to make the requests.

Name in the server's certificate must be used by domino-db clients

The domino-db module connecting to Proton must use a dns name or ip address that exists in the server's certificate.

Proton will report the following error when the client is attempting to connect to the server using a hostname name or address that is not on the server's certificate:

PROTON> No match found for server name: xxxxxxxxxxxxxxx.

Where xxxxxxxxxxxxxxx is the name/address the client used.

Configuration variables for troubleshooting

The following notes.ini variables are available to help developers and administrators view more details about the work that is being done by Proton. These are provided for troubleshooting and debugging purposes only. The amount of information that's displayed may change as needed in future releases.

PROTON_TRACE_REQUEST=1 produces request level information from applications. The information includes the type of request, number of documents, time to execute.
PROTON_TRACE_SESSION=1 produces information related to the identity of the client application.
PROTON_TRACE_SEARCH=1 produces infomation related to search operations.
PROTON_TRACE_SESSION_CACHE=1 produces information related to session caching.

Domino AppDev Pack

1.0.3

Configure Domino keyring

Configure Proton Test Utility

Configure IAM Service

Configure Act-As-User

Configuration Test