Domino AppDev Pack

1.0.2

domino-db Advanced Topics

This page describes advanced topics of domino-db.

Compute with form

When you create, read or update a document it is sometimes useful to compute document items based on a form stored in the database. For example, consider a Contact form with FirstName and LastName items. The Contact form might have an input validation formula on LastName like this:

@If( LastName = ""; @Failure( "You must enter a last name" ); @Success )

When you use domino-db to create a Contact document, you can choose to compute-with-form before the document is saved to the database. Here's a trivial example of a request that should fail to create a document:

  const result = await database.bulkCreateDocuments({
    documents: [
      {
        Form: 'Contact',
        FirstName: 'Joe',
      },
    ],
    computeOptions: {
      computeWithForm: true,
    },
  });

Since there is no LastName item sent to the server the request should fail. To be precise, the BulkResponse object should contain a documents array with one element and that element should include an @error property indicating a validation error occurred. (Of course, this is true only if the Contact form has the necessary input validation formula.)

Sometimes it also makes sense to compute-with-form on a read operation. Let's say the Contact form also includes a computed-for-display item called FullName. The formula for FullName is a concatenation of two other items:

FirstName + " " + LastName

The FullName field is not actually stored on the document, but you can read the item like this:

  result = await database.bulkReadDocumentsByUnid({
    unids,
    itemNames: ['FullName'],
    computeOptions: {
      computeWithForm: true,
    },
    onErrorOptions: onError.CONTINUE,
  });

Because computeOptions.computeWithForm is true, the response should include any computed-for-display items include in the itemNames array.

The syntax for the computeOptions object is the same for all create, read and update requests. However, keep in mind that computed-for-display items are calculated only on read operations.

computeOptions {Object}

  • computeWithForm {boolean} -- true if you want to compute items based on a form stored in the database. The default is false.
  • ignoreComputeErrors {boolean} -- true if you want to continue processing a document with a compute error. This option has no effect unless computeWithForm is also true. The default is to stop processing a document with a compute error.

Obviously, this is a relatively advanced topic. Your compute results will vary based on the design of forms stored in your target database. For more information about computed items see this IBM Domino Designer topic

Query arguments

When you construct a query to use with domino-db it often makes sense to define the query exactly. For example, the following query finds all contact documents where LastName is Aardman:

Form = 'Contact' and LastName = 'Aardman'

On the other hand, you sometimes want to construct a query with variables and then substitute those variables at run time. For example, the following query defines a single variable (ln), the value of which is unknown until run time:

Form = 'Contact' and LastName = ?ln

And the following sample shows how you might use such a query string in a call to bulkReadDocuments:

const { useServer } = require('@domino/domino-db');
const serverConfig = require('./server-config.js');
const databaseConfig = require('./database-config.js');

useServer(serverConfig).then(async server => {
  const database = await server.useDatabase(databaseConfig);
  const documents = await database.bulkReadDocuments({
    query: "Form = 'Contact' and LastName = ?ln",
    queryArgs: [
      {
        name: 'ln',
        value: lastNameValue,
      },
    ],
  });

  // documents is an array of documents -- one for each
  // document that matches the query
});

NOTE: Argument substitution is a core feature of the Domino Query Language (DQL). It's important to understand the substitution happens on the Domino server while Proton is processing your request.

Query arguments can be either named, as in the previous example, or unnamed. When unnamed, you substitute variables by ordinal position in the query. The following call to bulkReadDocuments is essentially equivalent to the previous one:

const documents = await database.bulkReadDocuments({
  query: "Form = 'Contact' and LastName = ?",
  queryArgs: [
    {
      ordinal: 1,
      value: lastNameValue,
    },
  ],
});

You can specify a queryArgs array when calling bulkReadDocuments or any other function that requires a query string. Each element of the array must be either a value or an object with the following properties:

  • ordinal {number} Required if no name is specified. The value of ordinal is the one-based position of the corresponding unnamed variable in the query string.
  • name {string} Required if no ordinal is specified. The value of name must match one of the named variables in the query string. For example, to match variable ?ln, specify ln.
  • value {string | number | Object} Required. This is the value to substitute for the matching variable.

The following example defines a query string and a queryArgs array with three elements:

  query: 'LastName = ? and Number = ? and Date = ?',
  queryArgs: [
    {
      ordinal: 1,
      value: stringValue,
    },
    {
      ordinal: 2,
      value: numberValue,
    },
    {
      ordinal: 3,
      value: {
        type: 'datetime',
        data: datetimeValue,
      }
    },
  ],

Even more simply, you can imply the ordinal position of each argument like this:

  query: 'LastName = ? and Number = ? and Date = ?',
  queryArgs: [
    stringValue,
    numberValue,
    {
      type: 'datetime',
      data: datetimeValue,
    },
  ],

Let's assume you use the above query and queryArgs properties in a call to bulkReadDocuments. Before executing the query on the server, DQL should substitute the arguments as follows:

  • Assuming stringValue is a string, DQL substitutes query variable 1 with the TYPE_TEXT value.
  • Assuming numberValue is a number, DQL substitutes variable 2 with the TYPE_NUMBER value.
  • Assuming datetimeValue is a valid ISO8601 date string, DQL substitutes variable 3 with a TYPE_TIME value.

If you specify a query argument value of a different type (e.g. boolean or undefined), you should expect the operation to fail.

Attachments

Reading attachments

Whether you are using Document::readAttachmentStream(), Database::bulkReadAttachmentStream() or Database::bulkReadAttachmentStreamByUnid(), domino-db returns a promise that resolves to a readable stream. For example, consider the following code excerpt:

    const readable = await document.readAttachmentStream({
      fileNames: ['photo_1.jpg', 'photo_2.jpg'],
    });

In the above example, readable is a stream object used to read the contents of two attachments. To begin reading the attachments, you should register listeners for the following stream events:

    readable.on('file', file => {
      // A file attachment is starting
    });
    readable.on('data', data => {
      // A chunk of attachment data has arrived
    });
    readable.on('eof', () => {
      // The file attachment has ended
    });
    readable.on('end', () => {
      // The stream has been closed
    });
    readable.on('error', e => {
      // An error occurred
    });

NOTE: While a domino-db attachment stream may appear to be an instance of the Node.js Stream class, it is really an instance of EventEmitter. Currently, you cannot pipe an attachment stream, but you can use other functions shared by EventEmitter and Stream.

Event 'file'

The 'file' event is emitted each time the start of a new attachment arrives from the server.

    readable.on('file', file => {
      // A file attachment is starting
    });

The file argument is an object with the following properties:

  • unid {string} The UNID of the associated document.
  • fileName {string} The attachment file name.
  • fileSize {number} The attachment size in bytes.
  • modified {object} The last modified date of the attachment.
  • error {object} An error object. This property is set only when an attachment level error occurs (rare).

Event 'data'

The 'data' event is emitted each time a chunk of attachment data arrives from the server.

    readable.on('data', data => {
      // A chunk of attachment data has arrived
    });

The data argument is a Uint8Array of attachment data. This is a variable length buffer of raw bytes. Depending on the size of the attachment, there might be many 'data' events between a pair of 'file' and 'eof' events.

Event 'eof'

The 'eof' event is emitted when an individual file is finished.

    readable.on('eof', () => {
      // The file attachment has ended
    });

When Proton is streaming multiple attachments, the 'eof' event might be followed by another 'file' event.

Event 'end'

The 'end' event is emitted once when Proton has stopped sending data and the stream is closed.

    readable.on('end', () => {
      // The stream has been closed
    });

Event 'error'

The 'error' event is emitted only when there is a fatal error at the protocol level.

    readable.on('error', e => {
      // An error occurred
    });

The e argument is an instance of the DominoDbError class. For example, if domino-db is unable to connect to Proton, the 'error' event is emitted once and the stream is closed.

Read example

Consider the following example:

    const readable = await document.readAttachmentStream({
      fileNames: ['photo_1.jpg', 'photo_2.jpg', 'photo_3.jpg'],
      chunkSizeKb: 32,
    });
    readable.on('file', file => {
      // A file attachment is starting
    });
    readable.on('data', data => {
      // A chunk of attachment data has arrived
    });
    readable.on('eof', () => {
      // The file attachment has ended
    });
    readable.on('end', () => {
      // The stream has been closed
    });
    readable.on('error', e => {
      // An error occurred
    });

Let's say the associated document includes two out of the three requested attachments. It includes photo_1.jpg and photo_3.jpg, but NOT photo_2.jpg. The expected result is:

  • The 'file' event is emitted once for photo_1.jpg.

  • The 'data' event is emitted N times for photo_1.jpg. If the entire file fits in a single chunk, the event is emitted only once for photo_1.jpg. Otherwise, the event is emitted multiple times for photo_1.jpg.

  • The 'eof' event is emitted once for photo_1.jpg.

  • Zero events are emitted for photo_2.jpg because the attachment doesn't exist.

  • The 'file' event is emitted once for photo_3.jpg.

  • The 'data' event is emitted N times for photo_3.jpg.

  • The 'eof' event is emitted once for photo_3.jpg.

  • After all the other events are complete, the 'end' event is emitted once indicating the stream has closed.

Writing attachments

When you use Database::bulkCreateAttachmentStream(), domino-db returns a promise that resolves to a writable stream. For example, consider the following code excerpt:

    const writable = await database.bulkCreateAttachmentStream({});

In this example, writable is a stream object used to write the contents of one or more attachments. To begin writing the attachments, you should first register listeners for the following stream events:

    writable.on('response', response => {
      // The attachment content was written to the document(s) and a
      // response has arrived from the server
    });
    writable.on('error', e => {
      // An error occurred and the stream is closed
    });

For details, see the writable event descriptions beginning with Event: 'response'.

After registering your listeners, you use the file() and write() functions to write your data. And you use the end() function to close the stream.

    writable.file({
      unid: '49CDF4368D68B2C185258359007B465C',
      fileName: 'example.txt',
    });
    writable.write(new Uint8Array([97, 98, 99, 100, 101, 102, 103])); // abcdefg
    writable.write(new Uint8Array([65, 66, 67, 68, 69])); // ABCDE
    writable.end();

NOTE: While a domino-db attachment stream may appear to be an instance of the Node.js Stream class, it is really an instance of EventEmitter. Currently, you cannot pipe to a writable attachment stream, but you can use other functions shared by EventEmitter and Stream.

Event: 'response'

The 'response' event is emitted once after all attachment content has been written to the server.

    writable.on('response', response => {
      // The attachment content was written to the document(s) and a
      // response has arrived from the server
    });

The response argument is an object with a single attachments property -- itself an array of objects. Each object in the attachments array has the following properties:

  • unid {string} The UNID of the associated document.
  • fileName {string} The attachment file name.
  • fileSize {number} The size of the attachment.
  • modified {object} The last modified date of the attachment.

Event: 'drain'

The 'drain' event is emitted after the write stream has drained its internal buffer and is ready to accept more data.

    writable.on('drain', () => {
      // The write stream has drained and you may safely
      // write more data
    });

It's especially important to listen for this event when you are writing a large attachment. For more information on the 'drain' event, see Write stream draining example.

Event: 'error'

The 'error' event is emitted only when an error occurs.

    writable.on('error', e => {
      // An error occurred and the stream is closed
    });

The e argument is an instance of the DominoDbError class. For example, if domino-db is unable to connect to Proton, the 'error' event is emitted once and the stream is closed.

writable.file()

Marks the beginning of a new attachment in the stream.

    writable.file({
      unid: '49CDF4368D68B2C185258359007B465C',
      fileName: 'example.txt',
    });

This function accepts a single object with the following properties:

  • unid {string} The UNID of the target document.
  • fileName {string} The attachment file name.

Both properties are required.

writable.write()

Writes a chunk of data to the stream.

    writable.write(new Uint8Array([97, 98, 99, 100, 101, 102, 103])); // abcdefg
    writable.write(new Uint8Array([65, 66, 67, 68, 69])); // ABCDE

This function accepts an instance of Uint8Array containing a chunk of attachment data. Especially when writing a large attachment, you should write the data in reasonable size chunks (e.g. 32 kilobytes).

The write() function returns a boolean value indicating whether the stream is successfully draining its internal buffer. A value of true indicates the stream buffer is draining. A value of false indicates the internal buffer has reached its high water mark. For more details, see Write stream draining example.

writable.end()

Closes the stream.

    writable.end();

This function has no arguments.

Write example

Consider the following example:

    const writable = await database.bulkCreateAttachmentStream({});
    writable.on('error', e => {
      // An error occurred and the stream is closed
    });
    writable.on('response', response => {
      // The attachment content was written to the document and a
      // response has arrived from the server
    });

    writable.file({
      unid: '49CDF4368D68B2C185258359007B465C',
      fileName: 'foo.txt',
    });
    writable.write(new Uint8Array([97, 98, 99, 100, 101, 102, 103]));
    writable.write(new Uint8Array([65, 66, 67, 68, 69]));
    writable.end();

The expected result is the 'response' event is emitted once, shortly after writable.end() returns. The response object is expected to be something like this (when serialized to JSON):

    {
      "attachments": [
        {
          "unid": "49CDF4368D68B2C185258359007B465C",
          "fileName": "foo.txt",
          "fileSize": 12,
          "modified": {
            "type": "datetime",
            "data": "2018-05-10T18:20:15.31Z"
          }
        }
      ]
    }

Of course, if the example is modified to write multiple attachments, the expected response would include more than one attachment.

Write stream draining example

The previous example wrote a trivially small attachment to a single document. When you write a large attachment or even several medium size attachments, you should be aware of the write stream's internal buffer. When the buffer doesn't drain quickly enough, it can be dangerous to write more data to the stream.

The following example assumes writable is a domino-db attachment stream and buffer is a large Buffer of binary data. It writes the data to a single attachment in 16 kilobyte chunks:

    let error;
    writable.on('error', e => {
      error = e;
    });

    writable.on('response', response => {
      // The attachment content was written to the document and a
      // response has arrived from the server
    });

    // Write the image in n chunks
    let offset = 0;
    const writeRemaining = () => {
      if (error) {
        return;
      }

      let draining = true;
      while (offset < buffer.length && draining) {
        const remainingBytes = buffer.length - offset;
        let chunkSize = 16 * 1024;
        if (remainingBytes < chunkSize) {
          chunkSize = remainingBytes;
        }

        const chunk = new Uint8Array(
          buffer.slice(offset, offset + chunkSize),
        );

        draining = writable.write(chunk);
        offset += chunkSize;
      }
      if (offset < buffer.length) {
        // Buffer is not draining. Write some more once it drains.
        writable.once('drain', writeRemaining);
      } else {
        writable.end();
      }
    };

    writable.file({ unid, fileName });
    writeRemaining();

In this example writeRemaining() is a local function. If the stream's internal buffer is draining, writeRemaining() is called only once. Otherwise, it pauses until the stream is drained. It uses writable.once() to listen for the 'drain' event:

        // Buffer is not draining. Write some more once it drains.
        writable.once('drain', writeRemaining);

The stream calls writeRemaining() back when the 'drain' event fires. Actually the stream may call writeRemaining() several times until all the data is written. At that point, writeRemaining() calls writable.end() and the 'response' event should fire.

Deleting attachments

To delete attachments, you use Document::deleteAttachments(), Database::bulkDeleteAttachments() or Database::bulkDeleteAttachmentsByUnid().

NOTE: There are two kinds of attachments in Notes and Domino. A normal attachment is associated with a rich text item and appears as a hot spot in the Notes client. A V2 style attachment is not associated with a rich text item and appears "below the line" in the Notes client. The above domino-db functions work best with V2 style attachments. When you use domino-db to delete a normal attachment, it does not delete the associated rich text hot spot.

← domino-db SecurityProton Test Script →