queue.name = Queue 1
Report Generator - Administration Guide : Distribution
04 April 2016
1. Distributing the Report Creation
Creating a report is typically a resource demanding task in terms of memory use and CPU usage, this because a report typically extracts large amount of data and uses XSLT transformation to convert XML. Even though the internals of the report generator has been implemented in a very efficient way, there will always be a cost in terms of performance to create reports. The report generator has a solution to this, by providing support for distributed creation of reports through use of so called Agents. The agents can be distributed across several computers in the network.
Within the ENOVIA database, you can create business objects of type "TVC Queue".
A report can be associated with a particular queue, by setting the "queue.name" property within the "TVC Conversion Properties" attribute to the name of the "TVC Queue" object. Example:
This would correspond to the business object of type "TVC Queue" with name "Queue 1" and having the revision set to empty. The object must be created inside the "TVC Administration" vault.
Whenever a report is created, and the "queue.name" property is defined, a job object will be created and connected to the queue object.
Each queue that is in use needs a so called Agent, which is a separate process that is polling the queue for new jobs. It is possible to have several Agents attached to the same queue; however, one queue agent can only be attached to one queue. This is also illustrated in the picture below.
Figure 1. Queue and Agent
1.1. Data Model
Within the database, objects of type TVC Queue represent the queues. The information required to create a report is stored inside an object of type "TVC Job". Job objects are connected to a queue with the relationship "TVC Job".
As of TVC 2011.2, the Agents can also be represented in the database as objects (this is not required though). Within the start script of an agent, you can define an identifier for the agent and if present an agent object will also be created inside the database. The job object will in such case also be connected to the Agent that is processing the job.
To summarize, the data model around this looks like:
The relationship between the Queue and Job and between the Agent and Job is called "TVC Job". The connection between the Agent and Queue is called "TVC Agent". Neither of these relationship types has any attributes.
1.1.1. Attributes
The table below illustrates the attributes being used.
| Business Type | Attribute Name | Purpose | ||
|---|---|---|---|---|
TVC Agent |
TVC Agent Last Alive |
A time stamp indicating the time when the agent last was processing a job. |
||
TVC Agent Startup |
The time stamp when the agent were started up |
|||
TVC Agent Stop Request |
A Boolean attribute that can be set to instruct the Agent to terminate when all ongoing jobs has been completed. |
|||
TVC Queue |
TVC Keep Jobs |
A Boolean value indicating if the jobs should be deleted or not. Setting this attribute to true allows accessing the generated reports after completion. The user can from the user interface see his/her created reports and download them for example. |
||
TVC Keep Time |
A string indicating how long time a job is/should be kept after completion. The value of this attribute is a string, and the format is:
Examples:
Scripts are available for cleaning up jobs that are older than the configured keep time. Such script can be setup to be ran periodically.
|
|||
TVC Job |
TVC Job Status |
Contains a string used to track the status of a completed job. This attribute is used internally by the report generator and should not be manipulated. |
||
TVC Job Info |
Contains a string that is used to identify the job. The value is populated based upon the conversion property "jobInfoMacro". |
|||
TVC Cancel Requested |
A boolean value that indicates that the job should be cancelled. E.g. when the agent takes this job, the job is immediately cancelled. |
1.1.2. Lifecycles
The types Queue, Agent and Job have its own policies. These are described in the table below:
| Business Type | Policy | States | Description |
|---|---|---|---|
TVC Queue |
TVC Queue |
Exists |
N/A |
TVC Agent |
TVC Agent |
Exists |
N/A |
TVC Job |
TVC Job |
Created |
Initial state. |
Waiting |
When the job is connected to a queue, the job is promoted to this state. The job is cancellable in this state. |
||
In Work |
When an agent starts processing a job, the job is promoted to this state. |
||
Done |
When the job has been completed, this state is reached. If the report failed or was cancelled, this is indicated within the TVC Job Status attribute on the job object. In case of success, the job object will also hold the generated report. |
1.2. The Queue Agent
The queue agent is a Java application connected to the ENOVIA/Matrix database. To start an agent one will use the provided start scripts (Windows and UNIX shell scripts available) with some modifications. The modifications required are defined in the sub chapters below.
1.2.1. Queue Agent Start Script
When the report generator is installed, you will have a folder under the WEB-INF directory in the web-application called "reportgenerator". This folder contains a Windows start script and a UNIX start script. These scripts are called:
QueueAgent.sh QueueAgent.bat
The script has been made as generic as possible and to use it, you will typically invoke the script with the following arguments:
| Argument | Description |
|---|---|
1 |
The name of the queue to bind to. This will set the variable QUEUE_NAME. |
2 |
The name of the agent. This will set the AGENT_ID variable. |
3 |
The action to perform. Any of:
These actions are described later |
| You need to modify the script slightly in order to get it working in your environment correctly. This is described in the next chapter. |
1.2.2. Required Script Changes
The first things you need to modify within these scripts are the following variables:
| Variable Name | Description | ||
|---|---|---|---|
WEBAPP_ROOT_DIR |
Points out where your web application is installed. This is needed as we load the JAR files and other resources dynamically from this directory. The value can be a relative or absolute path. Since the scripts are
located under If you place the script somewhere else, you must change this variable. |
||
CONTEXT_HOST |
Specifies how to connect to ENOVIA/Matrix. If you use RIP mode, this variable is empty, while if RMI mode is used this variable will point out a RMI server. Example:
or
|
Example:
SET WEBAPP_ROOT_DIR=d:\apps\apache-tomcat-6.0.29\webapps\enovia
SET CONTEXT_HOST=rmi://127.0.0.1:1099If you are running in RIP mode, you will need some additional Matrix related settings also in the script. However, it is out-of-scope in this document to describe the Matrix details.
On UNIX you can modify the MX_ENV_SCRIPT variable to point to a valid mxEnv.sh script.
Java Home
In the Windows start script, it is assumed that the JAVA_HOME variable is set. If not, this has to be set before invoking the script. For UNIX, you need to modify the script and define the JAVA home.
Defining the Number of concurrent Threads
Each queue agent can run X number of concurrent threads. Default this is set to 5. If needed, this value can be changed. Keep in mind that increasing the value requires more resources, like heap size etc. If the number of threads is too high, consider adding a new queue agent process.
The number of threads is specified by the parameter: THREADS.
1.2.3. Agent Actions
The available values for the action argument provided to the script are defined in the table below:
| Action | Description |
|---|---|
run |
|
start |
|
status |
Prints out some statistics for the queue such as how many jobs are waiting. |
stopall |
Requests all agents connected to the queue to stop. |
validate |
Validates how many and what jobs that can be deleted through the cleanup |
cleanup |
Cleans up old jobs that are older than configured keep-time |
1.3. Queue/Job Administration
If you have configured your queues to keep the jobs, you can use enable a command in the user interface allowing administrators to manage and inspect the jobs in the queues.
The URL to the admin console should be:
${ROOT_DIR}/tvc-action/rgAdminConsole
This URL can only be accessed by a person having either system or business privileges or having the role Administration Manager. Once this interface is shown, you will see something like:
Figure 2. Administration of Jobs
This page gives a tab for each queue, and each tab shows information about a queue such as some statistics and information about attached agents as well as all jobs related to a queue.
Through the list of jobs, you can inspect the jobs and depending on the status of a job, you can for example download the generated report, cancel a report that has not been started and delete a completed job.
Also, the administrator can request all agents to terminate (they will then terminate when all ongoing jobs are completed). Additionally, the administrator can run the clean up, e.g. delete all completed jobs that are older than the configured keep time.
1.4. Allowing a User to Access Jobs
If you have configured your queues to keep the jobs, you can use enable a command in the user interface allowing the users to access jobs that he/she has created. This in order to be able to download reports that has been created in the past time.
The URL to the user job console should be:
${ROOT_DIR}/tvc-action/rgUserConsole
When this page is opened, the user will see a table containing the jobs like shown below:
Figure 3. User jobs
From this page, the user can download reports that have been created or request jobs not yet being processed for cancellation.