Tracking Changes
An agent can keep track of the latest changes that have been made to extracted data.
The agent will mark extracted data as deleted, updated or added. If data is deleted but later returned in a modified state, the data will be marked as "reappeared".
An agent can also be configured to only export data that has changed since the last successful run or only export data that has changed since a specified number of days.
Data will be marked as deleted if it was extracted last time the agent ran, but not during the current run. Data will only be marked as deleted if an agent completes successfully. This prevents data from being incorrectly marked as deleted if an agent fails halfway through a run.
Default change tracking can be enabled on the Export container command which is available on the command menu in Sequentum Cloud.
The following advanced change tracking options are available:
Operation | Description |
Enable change tracking | Enables or disables change tracking. |
Track deletes | Specifies if an agent should track deleted data. If an agent does not track deleted data, the last change status will not change for data that was not found in the last successful agent run. If an agent is tracking deleted data, the last change status will be set to Deleted for data that was not found in the last successful agent run. |
Export | All Data - Exports all data no matter if the data has changed or not. Changed Data Only - Exports all data that has changed since the last successful run. |
Unique data identification | Determines the object identifier used in the internal database. This value uniquely identifies the data row and will not change unless the internal database is recreated. The name of the data column where the object identifier is stored. |
Last Change | Exports the type of change that was last made to a data row. The name of the data column where the type of change is stored. The following are values that are possible in this column: Added - A data row has been added. Updated - A data row has been modified from its previous run. Deleted - A data row has been deleted. Reappeared - A data row that was once marked as deleted has been seen with some columns updated. |
Columns Effected | Exports a column that contains the names of columns affected by a change. The name of the data column where the columns affected value is stored. |
Change Date | Exports the date a data row was last changed. The name of the data column where the change date is stored. |
Insert Date | The name of the data column where the insert date is stored. Exports the date a data row was first inserted. This is the date the data was first extracted. |
Last Seen | A data row will be marked as deleted if it hasn't been seen in the past number of days specified. The default value of 0 will mark the data row as deleted if the data row is not found in the next run. |
Changed Last Run | The name of the data column to store the value indicating if a data row changed the last time an agent was run. Exports a value indicating if a data row changed last time the agent was run. |
Unique data identification
When an agent exports data, it compares extracted data with existing data to see if the data has been added, changed or deleted. In order to see if a data entry has changed, the agent needs to be able to uniquely identify a data entry. By default, an agent uses all the captured data to identify a data entry, but this means that every time any data changes, a data entry is always identified as a new data entry because it never matches any existing data entry.
To get the change tracking working properly, the agent correctly identifies modified data entries and it's important to mark capture commands that extract data that uniquely identifies a data entry.
Reset Change Tracking
In order to reset change tracking, a user gets the option in the ‘Info’ tab (refer attached snapshot) on the Agent Control Center.
