FORGEBOX Enterprise 🚀 - Take your ColdFusion (CFML) Development to Modern Times! Learn More...

PresideCMS Extension: Preside Data API

v3.5.8+0000180 Preside Extensions

Preside Data API Extension

The Data API extension provides developers with the power to rapidly develop APIs against their Preside data models. With a few simple object annotations and optional i18n resource keys, developers are able to provide full CRUD APIs to the data model with beautifully rendered API documentation.

Note: Version 3.0.0 requires Preside version 10.11.0 and above.

Accessing the API

The default root URI of the API is /api/data/v1/ (see Namespaces and multiple APIs, further below). OpenAPI v3 specification can be browsed at /api/data/v1/docs/spec/ and HTML documentation based on the spec can be found at /api/data/v1/docs/html/ (or /api/data/v1/docs/swagger/, if you prefer).

Feature flags

As of v3.3.0, two feature flags are available that bring backward compatibility from previous versions. New default behaviour is to return null values for empty strings and numbers. Previously, the extension would return an empty string, "". To get the old behaviour back, you may set the following features to false:

settings.features.dataApiUseNullForNumerics.enabled = false;
settings.features.dataApiUseNullForStrings.enabled  = false;

Configuring your entities

Object annotations

The bare minimum configuration to enable an object for the API is to set @dataApiEnabled true:

/**
 * @dataApiEnabled true
 *
 */
component {
	// ...
}

Additional optional annotation options at the object level are:

  • dataApiEntityName: Alternative entity name to use in API urls, e.g. instead of crm_contact, you may wish to use contact.
  • dataApiCategory: For the HTML documentation. Allows sub-groups of entities. Especially useful for large APIs.
  • dataApiQueueEnabled: Whether or not the change queue is enabled for this object
  • dataApiQueue: Specific queue name for this object
  • dataApiSortOrder: Sort order for paginated results. Default is date last modified ascending.
  • dataApiSavedFilters: Comma-separated list of saved filters to apply to all requests to this object (e.g. only return active records)
  • dataApiIgnoreDefaultFilters: Comma-separated list of ignore default filters to apply to all requests to this object
  • dataApiVerbs: Supported REST HTTP Verbs. If not supplied, all verbs and operations are supported (i.e. GET, POST, PUT and DELETE)
  • dataApiFields: Fields to return in GET requests (defaults to all non-excluded fields)
  • dataApiUpsertFields: Fields to accept in POST/PUT request (defaults to dataApiFields)
  • dataApiExcludeFields: Fields to exclude from GET API calls
  • dataApiUpsertExcludeFields: Fields not to accept in POST/PUT requests (defaults to dataApiExcludeFields)
  • dataApiFilterFields: Fields to allow as simple filters for paginated GET requests (defaults to foreign keys, boolean and enum fields)
  • dataApiAllowIdInsert: Whether or not to allow the ID field to be set during a POST operation to create a new record
  • dataApiQueueRelevantFields: Fields that are relevant to trigger a record update to be queued. If not used, then any data change will be queued. If used then only the specified fields will be examined and in case of atomic-changes for the queue only the relevant field changes will be included in the queue item. Inserts and deletes are always queued, only during updates this annotation is evaluated.

Property annotations

Object properties support the following optional annotations:

  • dataApiAlias: An alternative name to use for the property in the API. i.e. instead of contact_status you may wish to just use status through the API.
  • dataApiRenderer: A non-default renderer to use when returning data through the API. See custom renderers, below.
  • dataApiDerivative: For image assets, specify the derivative to be used when rendering URLs.
  • dataApiType: For the documentation. The data type of the property, e.g. 'string'.
  • dataApiFormat: For the documentation. The format for the property, e.g. 'Email address'.
  • dataApiEnabled: Whether or not this property should be included in the API
  • dataApiUpsertEnabled: Whether or not this property should be included in the POST/PUT operations.

Custom renderers

If you specify a non-default renderer for an object property, it will be rendered using Preside's content rendering system. For example, the following property definition specifies a myCustomRenderer renderer:

property name="my_prop" dataApiAlias="myProp" dataApiRenderer="myCustomRenderer";

To implement this, you will need a corresponding viewlet at renderers.content.myCustomRenderer.dataapi or renderers.content.myCustomRenderer.default (a renderer context of dataapi will be used and the system will fallback to the default renderer if that context is not implemented).

For example:

// /handlers/renderers/content/MyCustomRenderer.cfc
component {

	private string function dataApi( event, rc, prc, args={} ){
		var value = args.data ?: "";

		return renderView( view="/renderers/content/myCustomRenderer/dataApi", args={ value=value } );
	}

}

Customizing documentation labels and descriptions

All of the labelling and text in the generated documentation can be found at /i18n/dataapi.properties and you should refer to that when customizing the default text. A bare minimum override might look like:

api.title=My Application's API
api.description=This is my awesome application's API and here is some general information about it.\n\
\n\
Team awesome xxx.
api.version=v2.0
api.favicon=data:image/x-icon;base64,iVBOR...

Categories

If you annotate your objects with a @dataApiCategory property, your categories can be documented with:

category.my_category.name=My Category
category.my_category.description=Markdown enabled description of my category.
category.my_category.sort.order=20

Object and field level customizations

The following set of .properties file keys can be added per object in your own dataapi.properties files to customize the documentation per entity/field:

# OBJECT LEVEL:

dataapi:entity.my_entity.name=My entities
dataapi:entity.my_entity.name.singular=My entity
dataapi:entity.my_entity.description=Description of my entity (or API description for this section of the docs)
dataapi:entity.my_entity.sort.order=10

dataapi:operation.my_entity.get.description=Description for the paginated GET operation for your entity
dataapi:operation.my_entity.get.200.description=Description for the successful (200) response documentation for paginated GET requests for your entity

dataapi:operation.my_entity.get.by.id.description=Description for the GET /{recordId}/ operation for your entity
dataapi:operation.my_entity.get.by.id.200.description=Description for the successful (200) response documentation for GET /{recordId}/ requests for your entity
dataapi:operation.my_entity.get.by.id.404.description=Description for the not found (404) response documentation for GET /{recordId}/ requests for your entity

dataapi:operation.my_entity.put.description=Description for the PUT / (batch update) operation for your entity
dataapi:operation.my_entity.put.body.description=Description for the http json body for the PUT / (batch update) operation for your entity
dataapi:operation.my_entity.put.200.description=Description for the successful (200) response documentation for PUT / requests for your entity
dataapi:operation.my_entity.put.422.description=Description for the validation failed (422) response documentation for PUT / requests for your entity
dataapi:operation.my_entity.put.by.id.description=Description for the PUT /{recordid}/ (single record update) operation for your entity
dataapi:operation.my_entity.put.by.id.body.description=Description for the http json body for the PUT /{recordid}/ (single record update) operation for your entity
dataapi:operation.my_entity.put.by.id.200.description=Description for the successful (200) response documentation for PUT /{recordid}/ requests for your entity
dataapi:operation.my_entity.put.by.id.422.description=Description for the validation failed (422) response documentation for PUT /{recordid}/ requests for your entity
dataapi:operation.my_entity.put.by.id.404.description=Description for the record not found (404) response documentation for PUT /{recordid}/ requests for your entity

dataapi:operation.my_entity.post.description=Description of the POST / (batch insert) operation for your entity
dataapi:operation.my_entity.post.body.description=Description for the http json body for the POST / (batch insert) operation for your entity
dataapi:operation.my_entity.post.200.description=Description for the successful (200) response documentation for POST / requests for your entity
dataapi:operation.my_entity.post.422.description=Description for the validation failed (422) response documentation for POST / requests for your entity

dataapi:operation.my_entity.delete.description=Description of the DELETE /{recordid}/ operation for your entity
dataapi:operation.my_entity.delete.200.description=Description of the successful (200) response documentation for DELETE /{recordId}/ operations for your entity

# Field level
dataapi:operation.my_entity.get.params.fields.my_field.description=Description for *filter* field
dataapi:entity.my_entity.field.my_field.description=Description of field

Further customizations using interceptors

The following interception points are used to allow you to more deeply customize the integration.

onOpenApiSpecGeneration

Fired when generating OpenApi v3 specification for the API. A spec struct will be present in the interceptorArgs that you can use to augment the specification.

preDataApiSelectData

Fired before selecting data through the API. Receives the following keys in the interceptData:

  • selectDataArgs: Arguments that will be passed to the selectData() call
  • entity: Name of the entity being operated on

postDataApiSelectData

Fired after selecting data through the API. Receives the following keys in the interceptData:

  • selectDataArgs: Arguments that were passed to the selectData() call
  • entity: Name of the entity being operated on
  • data: Rendered and prepared data that will be returned to the API caller

preDataApiInsertData

Fires before inserting data through the API. Receives the following keys in the interceptData:

  • insertDataArgs: Arguments that will be passed to the insertData() call
  • entity: Name of the entity being operated on
  • record: The data that will be inserted (struct)

postDataApiInsertData

Fires after inserting data through the API. Receives the following keys in the interceptData:

  • insertDataArgs: Arguments that were passed to the insertData() call
  • entity: Name of the entity being operated on
  • record: The data that will be inserted (struct)
  • newId: Newly created record ID

preDataApiUpdateData

Fires before updating data through the API. Receives the following keys in the interceptData:

  • updateDataArgs: Arguments that will be passed to the updateData() call
  • entity: Name of the entity being operated on
  • recordId: ID of the record to be updated
  • data: The data that will be inserted (struct)

postDataApiUpdateData

Fires after updating data through the API. Receives the following keys in the interceptData:

  • updateDataArgs: Arguments that were passed to the updateData() call
  • entity: Name of the entity being operated on
  • recordId: ID of the record to be updated
  • data: The data that will be inserted (struct)

preDataApiDeleteData

Fires before deleting data through the API. Receives the following keys in the interceptData:

  • deleteDataArgs: Arguments that will be passed to the deleteData() call
  • entity: Name of the entity being operated on
  • recordId: ID of the record to be deleted

postDataApiDeleteData

Fires after deleting data through the API. Receives the following keys in the interceptData:

  • deleteDataArgs: Arguments that were passed to the deleteData() call
  • entity: Name of the entity being operated on
  • recordId: ID of the record to be deleted

Data Change Queue(s)

By default, they system enables a queue feature: settings.features.dataApiQueue. When enabled, API users can be subscribed, through the admin UI, to listen for data changes to all, or a number, of entities in the system.

Configuring the queue system

As of v3.0.0, you have the ability to configure multiple queue endpoints per API (see also, Namespaces and multiple APIs, below). This can be used to group entities into queues so that multiple different services can process the queues independently. The available settings per queue are:

  • pageSize: Number of records returned with each call to queue. Default is 1
  • atomicChanges: Whether or not the queue should return atomic changes. If true, each item in the queue will contain only the fields that have changed. If false, default, each item in the queue will contain the latest state of the record. Default is false.
  • returnTotalRecords: Whether or not to return X-Total-Records header with the total queue size when getting items from the queue (default is true)

Defining queues

Queues are defined in the Preside rest API configuration in Config.cfc. To add a queue definition to the default API:

settings.rest.apis[ "/data/v1" ].dataApiQueueEnabled = true;
settings.rest.apis[ "/data/v1" ].dataApiQueues = {
	  default    = { pageSize=100, atomicChanges=true } // the default queue
	, highvolume = { pageSize=1000, atomicChanges=false, returnTotalRecords=false }
};

Per object queue settings:

Annotate your preside object CFC with @dataApiQueueEnabled (default is true) and @dataApiQueue (default is default) properties. For example:

/**
 * @dataApiQueueEnabled false
 */

or

/**
 * @dataApiQueue highvolume
 */

Skip a data api queue:

To prevent data api queue to be created when you insert data into an object, you can set skipDataApiQueue = true as argument in the insertData function. For example:

$getPresideObject( "example_object" ).insertData(
	  data             = { example_field_name = "example_value" }
	, skipDataApiQueue = true
);

Namespaces and multiple APIs

Introduced in v2.0.0

By default, the API is exposed at /api/data/v1/. However, there will be occasions when you want to expose your data in different ways for different purposes. Or, if you are writing an extension, you may want to namespace your API so that it does not clash with any existing default API implementation. You can also use this feature to host multiple versions of your API concurrently.

With just a small amount of configuration, you can use all of the Data API's functionality in a separate, namespaced instance. First, configure the endpoints in your Config.cfc:

settings.rest.apis[ "/myGroovyApi/v1" ] = {
	  authProvider     = "dataApi"
	, description      = "REST API to expose data with an alternate structure"
	, dataApiNamespace = "myGroovyApi"
	, dataApiQueueEnabled = true
	, dataApiQueues       = {
		default = { pageSize=100, atomicChanges=true }
	  }
};
settings.rest.apis[ "/myGroovyApi/v1/docs" ] = {
	  description      = "Documentation for myGroovyApi REST API (no authentication required)"
	, dataApiNamespace = "myGroovyApi"
	, dataApiDocs      = true
};

A few things to note here:

  • The key within settings.rest.apis (e.g. /myGroovyApi/v1) is the base URI for the API. This will have /api prepended to it in the full URL.
  • dataApiNamespace is the namespace for the alternate API, and will be used when configuring objects. This will usually be the first part of the URI, but does not need to be.
  • dataApiDocs marks that this is the endpoint for the Swagger documentation. This whole endpoint could be omitted if you do not require the automatic document generation.
  • Use authProvider to mark an endpoint as needing authentication. If you omit this, the resource will not require authentication. In the API Manager in Preside, you can allow users to have access to individual APIs - so a user could have access to /api/myGroovyApi/v1 but not to the default /api/data/v1, if you wish.

Annotations

You can annotate your objects using exactly the same annotations as described above, but with :{namespace} appended. For example:

/**
 * @dataApiEnabled             true
 * @dataApiEnabled:myGroovyApi true
 * @dataApiVerbs:myGroovyApi   GET
 *
 */
component {
	property name="label" dataApiAlias:myGroovyApi="some_other_label";
}

This would enable this object both for the default /data/v1 API, and for your custom /myGroovyApi/v1 API. However, for myGroovyApi the object would only allow GET access, and the label field would be called some_other_label instead of label.

You could also use this to specify an alternate renderer for a property, e.g. dataApiRenderer:myGroovyApi="alternateRenderer".

Note that namespaces do not inherit any annotations from the default API. Any annotations must be made explicitly with the :{namespace} suffix.

Labelling and text

You can use all the same label and description customisations in your /i18n/dataapi.properties file as above; simply prefix the key name with {namespace}.. For example:

myGroovyApi.api.title=My Second API
myGroovyApi.api.description=This is an alternate API to my application

myGroovyApi.entity.my_entity.name=My alternate entity

Unlike annotations, i18n properties will cascade up to the defaults. So if you do not make any customisations, you will see all the default text.

Interceptors

All the same interceptors will run when actions are taken in a namespaced API. However, the interception point name will have _{namespace} as a suffix. Again, there is no fallback to the default interceptor, so any interceptors will need to be explicitly defined for your namespace. For example:

public void function preDataApiSelectData( event, interceptData ) {
	// action to take for preDataApiSelectData on the default API
}
public void function preDataApiSelectData_myGroovyApi( event, interceptData ) {
	// action to take for preDataApiSelectData on the myGroovyApi API
}

Configuring individual user access

Provided that your APIs use the 'dataApi' auth provider (default), you will be able to manage user access to the APIs through the admin interface.

  1. Navigate to: System > API Manager
  2. Find the API you want to manage & click 'Data API authentication'
  3. Click Add user
  4. Enter a user name (can lookup existing users also) and configure access to individual entities, their verbs and optionally queue access

Changelog

v3.5.8

  • Validated data not passed through on update single record - PUT

v3.5.7

  • Always return id field in entity GETs
  • Stop silent try/catch that could lead to inconsistent return form of atomic data api queue items

v3.5.6

  • Queue with atomic changes do not include default value or generated value fields for INSERT
  • Update twgit release version generator

v3.5.5

  • #73 Exit early from field rendering to avoid needless handler invocation

v3.5.4

  • #71 Remove unecessary helper methods
  • #72 Make HtmlDocumentationSyntaxHighlighter a singleton

v3.5.3

  • #70 API call returns all fields when only invalid fields provided in fields parameter

v3.5.2

  • #68 Fix for issue where background errors recorded when deleting records without an id field

v3.5.1

  • DATAAPI-16 - Pass complete record through to renderer

v3.5.0

  • #64 Change to way delete operations queue when there are outstanding insert/update operations in the queue
  • #67 New postValidateUpsertData interception point

v3.4.0

  • Time values now have default formatting of HH:mm

v3.3.2

  • #62- Fix issues with bad content renderers for data with page links

v3.3.1

  • #61 - Fix bad i18n keys being used for documenting field descriptions

v3.3.0

  • #58 - Return null for empty strings and empty numeric field types

v3.2.0

  • Add dataApiIgnoreDefaultFilters feature
  • Fix issue deleteData( filter=... ) would not queue deletes

v3.1.3

  • Add property’s name to data returned by field settings

v3.1.2

  • Pass through all args when using recordCountOnly=true to ensure saved/extra filters are included

v3.1.1

  • Add getMultipleRecords() method for embedding entities

v3.1.0

  • Introduce new annotation for objects to limit which fields will be watched for pushing changes into a change queue

v3.0.22

  • Explicitly pass args through to documentation view (Preside 10.14 compatibility)

v3.0.21

  • Ensure assets are compiled in build

v3.0.20

  • Move build and deploy to GitHub actions
  • Fix: include filters in pagination links

v3.0.19

  • Fix: If object name and the api entity name are different within one entity, no rights can be assigned in the API Manager

v3.0.18

  • Fix missed merge conflict

v3.0.17

  • Add some safety checking for non-existant entities when building permissions forms (fixes potential errors with custom namespace configurations)

v3.0.16

  • Building on v3.0.15, skipApiQueueWhenSkipSyncQueue now considered for all CRUD data api changes. Default value is now true

v3.0.15

  • skipApiQueueWhenSkipSyncQueue object decoration to use syncSynQueue intercept data to determine whether to skip the api change queue or not. You can sync object data without it being added to the api change queue.

v3.0.14

  • Fix for issue where there may be custom API configuration for objects that do not exist. Just ignore the configuration rather than raise errors.

v3.0.13

  • Fix for Queue Record Count not being returned for the default queue

v3.0.12

  • Fix for #43. PUT operations could fail when providing an unchanged field that had a unique index.

v3.0.11

  • Reapply accidentally removed code from v3.0.10 pull request.

v3.0.10

  • Add ability to bypass queuing a record in insertData() with skipDataApiQueue argument.

v3.0.9

  • #39 Add configuration option for disabling X-Total-Records on queue fetches

v3.0.8

  • Ensure DELETE from queue works in all different queue scenarios

v3.0.7

  • #38 Specifying fields in request should respect property alias

v3.0.6

  • #37 Add dataApiDerivative argument to properties for rendering image asset URLs

v3.0.5

  • #36 Fix error where DELETE /queue fails for batch delete on default queue (array of IDs in JSON body)
  • #35 Ensure that 404 is shown when API resource URI not found (was throwing 500 error)

v3.0.4

  • #34 Ensure deleting entries from the queue works with all default queues in all namespaces

v3.0.3

  • #33 Ensure all queue items are picked up for default queues

v3.0.2

  • #29 Apply saved filters when queueing changes to data

v3.0.1

  • #27 Use field aliases when documenting and using filter fields in paginated GET requests
  • #26 Fix for errors raised when calling API for entity whose primary key is numeric

v3.0.0

Administration

  • Add admin UI for managing individual user access to entities, verbs and queues

Queue enhancements

  • Ability to turn off the queue feature entirely
  • Ability to turn off the queue feature per object
  • Allow queue to contain atomic data changes
  • Allow queue to return configurable number of records, rather than always 1
  • Allow multiple queues for different groups of objects and with individual queue settings

Documentation enhancements

  • Add new plain HTML documentation endpoint
  • Object properties should be able to specify type and format for their spec definitions
  • Ability to categorize entities (applies to HTML documentation only)
  • Ability to set sort order on entities
  • Ability to set favicon for an API
  • Use plural name of entity for default entity tag name

Miscellaneous

  • Automatically convert foreign keys to assets, links and pages to URLs.
  • Ability to exclude/include properties in the API through annotations on the properties

Bug fixes

  • Documentation of foreign keys: only use 'Foreign Key UUID' format when it really is a UUID (i.e., not for int)
  • Do not show authentication description when API does not use authentication

v2.0.2

  • Fix path to docs

v2.0.1

  • Better handling of namespaced routes and handler paths
  • Update queues to allow namespacing

v2.0.0

  • Add support for multiple configured APIs via namespacing
  • Add dataApiSavedFilters annotation

v1.0.19

  • Total record count should return filtered count if filter is applied

v1.0.18

  • Prevent stack overflow with recursive loading of services on application startup

v1.0.17

  • #2: Add preValidate interception point for upsert operations

v1.0.16

  • Fixes #1: ensure that booleans are rendered as 'true' or 'false', not '1' or '0'

v1.0.15

  • Add general error message documentation
  • When POSTing (creating records), do not ignore missing fields

v1.0.14

  • Add option for specifying whether or not ID field creation is supported

v1.0.5-1.0.13

  • Refactoring build system

v1.0.4

  • Change queue logic so that fetching next item from queue always returns the same item until the API user manually removes it from the queue

v1.0.3

  • Fix interceptor logic to not break deletes that use a filter other than record IDs

v1.0.2

  • Improve documentation of queue system API
  • Fix the possible field list for the 'fields' REST API param
  • Allow for default descriptions of commonly named fields

v1.0.1

  • Initial release

$ box install preside-ext-data-api

No collaborators yet.
  • {{ getFullDate("2018-11-08T04:35:36Z") }}
  • {{ getFullDate("2023-11-08T15:31:34Z") }}
  • 6,881
  • 103,136