TIF ENOVIA/3DExperience Connector - Reply Handler

A reply handler is responsible for handling asynchronous replies from other systems and update the status of the job (which is the source for the reply) inside TIF. This is a crucial mechanism if you are interested in receiving events upon completion or failure of an integration job.

A typical use case is when you transfer data to a messaging queue and you expect a reply from the "other" system onto a reply-queue, then you need some functionality that consumes messages from the reply-queue in order to update the job status in TIF; e.g. mark the job as succeeded or failed.

Currently, you may handle replies using a configurable reply handler from following sources:

File system: Support for receiving events from the OS when a file is changed/added in a directory may be used to communicate reply status.
JMS: Consume messages from a particular JMS queue used for replies
IBM/Native MQ: Consume messages from an IBM MQ queue used for replies
RabbitMQ/AMQP: Consume messages from a Rabbit MQ queue
Kafka: Consume messages from a Kafka topic

A reply handler will take care about most of the details regarding receiving the reply and perform the internal calls inside TIF but provide extension points were necessary. For example, allow evaluating if the reply represents a positive or negative response.

A reply handler works like this

Get/obtain message from some source
From the message and its source - map to the originating Job / Transfer
- This is typically done either via correlation id or through some custom evaluation
Based upon the content of the message, evaluate if the response is successful or failing
- This is the place were you will need to plugin some code that does this evaluation.
Update the TIF internally

Configuration Location

A reply handler is defined in an XML resource within the ${TIF_ROOT}/modules/enovia/cfg directory of type "replyhandler".

Example:

$\{TIF_ROOT}/modules/enovia/cfg/replyhandler/MyReplyHandler.xml
$\{TIF_ROOT}/modules/enovia/cfg/domain/replyhandler/MyReplyHandler.xml

these configuration files are referenced as

tvc:replyhandler/MyReplyHandler.xml
tvc:replyhandler:domain/MyReplyHandler.xml

Configuration Format

A configuration defines the following aspects:

The source
How the mapping between message and Job id is made
How to evaluate the status
If you need an ENOVIA/3DEXPERIENCE context available during the execution. (default none is allocated)

Let’s look at the configuration format. Note that this is not a valid example it just illustrates all configuration aspects.

<ReplyHandler>
    <Source>
        <JMS id="eco-reply" /> (1)
        <NativeMQ id="mq-eco-reply" />
        <RabbitMQ id="rabbitmq-eco-reply" />
        <Kafka id="kafka-reply" />
        <File id="file-in-1" replyFor="file-out-1"/>
    </Source>

    <JobIdLocator className="com.acme.custom.MyJobLocator" /> (2)
    <JobIdLocator script="MyIDLocator.js" />
    <JobIdLocator script="tvc:script/MyIDLocator.js" />
    <JobIdLocator script="tvc:script:domain/AnotherIDLocator.js" />
    <JobIdLocator>
        importClass(com.technia.tif.enovia.job.reply.config.JobIDLocator.ID);

        function locate(msg) {
            ...
            return new ID(UUID.fromString(jobId), transferId);
        }
    </JobIdLocator>

    <StatusEvaluator className="com.acme.custom.MyStatusEvaluator" /> (3)
    <StatusEvaluator script="MyStatusEvaluator.js" />
    <StatusEvaluator script="tvc:script/MyStatusEvaluator.js" />
    <StatusEvaluator script="tvc:script:domain/AnotherStatusEvaluator.js" />
    <StatusEvaluator>
        function evaluate(ctx) {
            ctx.setResult(true, "OK");
        }
    </StatusEvaluator>

    <WithContext /> (4)

</ReplyHandler>

1	Either one of these sources can be used per reply handler
2	The JobID locator
3	The StatusEvaluator
4	Optionally element that, if present, will allocate a context for you.

The <WithContext> element supports the following attributes.

Attribute	Required	Description
user	No	If omitted, the super user will be used
securityContext	No	Defines the security context to be used
useDefaultSecurityContext	No	If set to true, the default security context for the user will be used. The used user must have a default security context otherwise an error will be raised.

Attribute

Required

Description

user

If omitted, the super user will be used

securityContext

Defines the security context to be used

useDefaultSecurityContext

If set to true, the default security context for the user will be used.

The used user must have a default security context otherwise an error will be raised.

Both the job-id locator and status-evaluator may be implemented in a few different ways:

A Java class implementing either com.technia.tif.enovia.job.reply.config.JobIDLocator for JobIdLocator and com.technia.tif.enovia.job.reply.config.StatusEvaluator for StatusEvaluator.
A script stored inside its own file using the script resource type.
An inline script.

In some cases you may not need a JobID locator. See next chapter(s).

If you are using Java 8, the internal Java Script engine has been changed so you may need to change your old legacy scripts written prior to Java 8 by adding the following at the top of your script:

try{load("nashorn:mozilla_compat.js");}catch (e){}

JMS Source

The supported attributes on the JMS element are shown in the table below

Attribute Description Required

Attribute	Description	Required
id	The id of the corresponding destination	Yes
messageSelector	A message selector that filters messages. See below for details	No
transacted	Boolean defining transaction mode	No. Default is false
ackMode	One of "auto","client" or "dups_ok"	No. Default is auto
consumerCount	Defines number of concurrent consumers. Typically used with destination listening to a queue.	No. Default is 1
shareConnection	Boolean defining if to share connection per destination with other JMS listeners. The common default value can be configured with property `jms.listener.shareConnection` in `${TIF_ROOT}/modules/enovia/etc/module.custom.properties`	No

The id of the corresponding destination

Yes

messageSelector

A message selector that filters messages. See below for details

transacted

Boolean defining transaction mode

No. Default is false

ackMode

One of "auto","client" or "dups_ok"

No. Default is auto

consumerCount

Defines number of concurrent consumers. Typically used with destination listening to a queue.

No. Default is 1

shareConnection

Boolean defining if to share connection per destination with other JMS listeners.

The common default value can be configured with property jms.listener.shareConnection in ${TIF_ROOT}/modules/enovia/etc/module.custom.properties

For handling of replies from a JMS messaging systems the correlation id is used to correlate the message to its "source" in TIF.

If you have changed the format of the correlation id in the outgoing message, you need to apply changes in the reply handler.

By default, the message selector that is used for a JMS Source is set to

JMSCorrelationID like '${tif.instance.id}|%'

The macro is resolved at runtime to the id of the TIF instance.

This means that only messages matching this correlation id will be fetched for the TIF instance in question. This is particularly useful in situations where you have multiple TIF instances listening to the same queues. In such case it is important that only the TIF instance that sent the original message will receive the reply.

If you are using the default correlation id on the outgoing JMS message, you do not need to define a JobIdLocator at all since the mapping can be done automatically.

Example config for JMS source were response message is of type TextMessage and the status is provided as a property on the message object.

<ReplyHandler>
    <Source>
        <JMS id="eco-reply" />
    </Source>
    <StatusEvaluator>
        function evaluate(ctx) {
            var msg = ctx.getMessage().getText();
            var succeeded = ctx.getMessge().getBooleanProperty("success");
            ctx.setResult(succeeded, msg);
        }
    </StatusEvaluator>
</ReplyHandler>

Rabbit MQ Source

The supported attributes on the RabbitMQ element are shown in the table below

Attribute	Description	Required
id	The id of the corresponding destination	Yes

Attribute

Description

Required

The id of the corresponding destination

Yes

Replies from a Rabbit MQ messaging system uses the correlation id of the message to correlate the message to its "source" in TIF.

If you have changed the format of the correlation id in the outgoing message, you need to apply changes in the reply handler.

By default, outgoing messages from TIF to RabbitMQ uses a correlation id that contains the following information

TIF instance id
Job id
Transfer id

The TIF instance id is needed in order to be able to correlate messages to the correct TIF instance. E.g ensure that the same TIF instance that sent the original message will handle the reply.

If you are using the default correlation id on the outgoing Rabbit MQ message, you do not need to define a JobIdLocator at all since the mapping can be done automatically.

Example config.

<ReplyHandler>
    <Source>
        <RabbitMQ id="eco-reply" />
    </Source>
    <StatusEvaluator>
        function evaluate(ctx) {
            var msg = ctx.getMessage().getBodyAsString();
            var succeeded = ctx.getProperties().getHeaders().get("status");
            ctx.setResult(succeeded, msg);
        }
    </StatusEvaluator>
</ReplyHandler>

Native MQ Source

The supported attributes on the NativeMQ element are shown in the table below

Attribute Description Required

Attribute	Description	Required
id	The id of the corresponding destination	Yes
defaultMatchGroupId	Whether or not if to match on group id’s. Note that per default this attribute is based upon the TIF setting `nativeMQ.defaultUseGroupId`. See also this chapter. By default TIF will match messages based upon their group-id’s.	No
messageId	Used to specify match option on the message-id	No
correlationId	Used to specify match option on the correlation-id	No
groupId	Used to specify match option on the group-id	No
seqNumber	Used to specify match option on the sequence number	No

The id of the corresponding destination

Yes

defaultMatchGroupId

Whether or not if to match on group id’s.

Note that per default this attribute is based upon the TIF setting nativeMQ.defaultUseGroupId.

Kafka Source

The supported attributes on the <Kafka> element are shown in the table below

Attribute Description Required

Attribute	Description	Required
id	The id of the corresponding destination	Yes
topic	Specifies the Kafka topic to consume records from	Yes unless the mapped destination have a default topic defined.
clientId	A client identifier	No
groupId	Specifies the group identifier. This is used to join the Kafka consumer with a specific consumer group	Yes.
instanceIdHeader	Specifies the name of the header on the record we consume that will contain the TIF instance value. This is only needed in case you have multiple TIF instances that may consume records from the same topics. In such case you must consider the source of the record.	Depends
jobIdHeader	Specifies the name of the header on the record that will contain the TIF Job ID	Yes
destinationIdHeader	Specifies the name of the header on the record that will contain the TIF destination id, which the record is a reply for. Note that you can omit this header and instead specify a static value in the `replyFor` attribute.	Depends
replyFor	Specifies a fixed destination id value. All records consumer are considered being originated for this destination. Either this attribute or the `destinationIdHeader` must be defined.	Depends

The id of the corresponding destination

Yes

topic

Specifies the Kafka topic to consume records from

Yes unless the mapped destination have a default topic defined.

clientId

A client identifier

groupId

Specifies the group identifier. This is used to join the Kafka consumer with a specific consumer group

Yes.

instanceIdHeader

Specifies the name of the header on the record we consume that will contain the TIF instance value.

This is only needed in case you have multiple TIF instances that may consume records from the same topics. In such case you must consider the source of the record.

Depends

jobIdHeader

Specifies the name of the header on the record that will contain the TIF Job ID

Yes

destinationIdHeader

Specifies the name of the header on the record that will contain the TIF destination id, which the record is a reply for.

Note that you can omit this header and instead specify a static value in the replyFor attribute.

Depends

replyFor

Specifies a fixed destination id value. All records consumer are considered being originated for this destination.

Either this attribute or the destinationIdHeader must be defined.

Depends

Example config for Kafka source.

<ReplyHandler>
    <Source>
        <Kafka id="kafka-reply-dest"
               groupId="test-group"
               topic="plm-reply"
               jobIdHeader="jobId"
               replyFor="kafka-dest"  />
    </Source>
    <StatusEvaluator>
        function evaluate(ctx) {
            ....
            ctx.setResult(succeeded, msg);
        }
    </StatusEvaluator>
</ReplyHandler>

When TIF is running in development mode, you can edit, create or delete configurations at runtime and those will be hot deployed automatically without the need to restart the TIF instance or take any further action.

However, in production mode, you can only edit a definition at runtime, but in order to take the changes into use you need to restart the corresponding service activator from within the Administration UI. Add or delete configurations is not supported.

File Source

Some use cases involves creating files in folders that are watched by other applications. In order to receive status updates from such operation, you can use a File source to listen into a folder, which the other system is responding into.

<ReplyHandler>
    <Source>
        <File id="file-dest-2"
              replyFor="file-dest-1"/>
    </Source>
    <StatusEvaluator className="com.technia.tif.enovia.job.reply.config.SimpleFileStatusEvaluator" />
</ReplyHandler>

In this example we use the default ID locator, which assumes that the file name is the "job id" + an optional suffix.

In the destinations.xml file you may define the outgoing file destination like below in order to include the job-id in the outgoing file-name.

<Destinations>
    <File id="file-dest-1"
          directory="${tif.temp}/transfer/out"
          fileName="${job.id}.xml"/>

    <File id="file-dest-2"
          directory="${tif.temp}/transfer/in"/>
</Destinations>

Moreover, we use a built-in status evaluator that has a very simple implementation:

    public void evaluate(ReplyHandlerContext ctx) {
        Path p = ctx.getMessage();
        String dirName = p.getParent().getFileName().toString();
        boolean error = "error".equalsIgnoreCase(dirName);
        ctx.setResult(!error, readFileContent(p));
    }

Startup of Reply Handler

Upon startup, TIF will automatically find all reply handler configurations that you have configured and deploy these. This method is called auto-registration and can be disabled if wanted. Below is a table of properties that are of interest.

Property	Type	Default	Description
resources.replyHandler.autoRegister	boolean	True	Use this property to disable the auto registration
resources.replyHandler.excluded	Comma separated list		Comma separated list of resources to be excluded.
resources.replyHandler.included	Comma separated list		Comma separated list of resources to be included.

Property

Type

Default

Description

resources.replyHandler.autoRegister

boolean

True

Use this property to disable the auto registration

resources.replyHandler.excluded

Comma separated list

Comma separated list of resources to be excluded.

resources.replyHandler.included

Comma separated list

Comma separated list of resources to be included.

By default, auto registration is enabled and no resources are excluded.

In earlier versions of TIF, you were forced to specify the reply handler configurations to be started within the ${TIF_ROOT}/modules/enovia/etc/module.custom.properties file. This method is still supported, although the preferred approach is to auto-register all configurations without having to also do extra configuration within the "module.custom.properties" file.

To deploy a reply handler using the old approach, see instructions below:

Syntax of entry within ${TIF_ROOT}/modules/enovia/etc/module.custom.properties

replyHandler.<id>.<property> = <value>

Below is an example where two different reply handlers has been configured.

replyHandler.0.config = ECOReply.xml
replyHandler.1.config = tvc:replyhandler:domain/PartReply.xml

Refering to configurations in the default domain do not require the complete expanded name.

E.g. ECOReply.xml is expanded to tvc:replyhandler/ECOReply.xml

Each reply handler may be stopped / restarted from the TIF Administration UI.

The most common configuration requires only the "config" property to be defined. However, there are additional properties available as shown below:

replyHandler.<id>.enabled = true (1)
replyHandler.<id>.config = ... (2)
replyHandler.<id>.className = ... (3)

1	May be used to disable a particular reply handler
2	Specifies the reply handler configuration
3	Instead of a configuration, specify a Java class implementing `com.technia.tif.enovia.job.reply.ReplyHandler`

The earlier properties like shown below to control the ENOVIA/3DEXPERIENCE context settings, are no longer supported.

replyHandler.<id>.context
replyHandler.<id>.context.user
replyHandler.<id>.context.password

You should configure this from within the reply handler instead using the <WithContext> element.

See previous chapter.

Monitoring Async Replies

To get control of the replies that are expected to come back to TIF from the other system(s), you can enable a scheduled task called "Async Reply Supervisor".

This task will typically run every night and find transfers that lack replies and flag these to allow someone to take some action.

There are some configuration properties within the module.properties file that controls this scheduled task. See below for an example:

asyncReplySupervisor.enabled=true
asyncReplySupervisor.execute=0 10 2 * * ?
asyncReplySupervisor.overDueTime=7d
asyncReplySupervisor.overDueReport.recipients=tif-admin-1@acme.com;tif-admin-2@acme.com
asyncReplySupervisor.overDueAction=mark-failed
asyncReplySupervisor.overDueAction.delay=5d

The overdue-time is the time-threshold that we expect a reply to be delivered. If there are jobs having transfers fulfilling the criteria, the scheduled task will send out email to the recipients listed.

After sending the email notification, there will be a period of time, in this case 5 days, which the administrator has some possibility to take action. If the transfer after these 5 days still has not received any reply, the associated job will be marked as failed.