Getting Started

In this section, we'll cover the process of using Sequentum Cloud Agent Editor to create a very simple web data extraction agent so you can quickly familiarize yourself with the steps of writing, testing, publishing, scheduling, running, monitoring the runtime status and reviewing exported data. We'll work through each of these steps with examples matching common web data extraction scenarios, so you can get comfortable building your own agents. We have a separate tutorial for writing a more complicated agent here.

This section assumes that you have already created an account on Sequentum Cloud and followed the steps to login. See the Login and Dashboard page for information on those steps.

The following image shows a brief animation of the steps we will walk you through. Read below for a full explanation.

Topics covered in this guide:

Before you begin, you might want to check out the Sequentum Cloud Editor section of the manual for a quick tour of the layout and tools in the Editor screen.

Choosing a Start URL

To begin, login to Sequentum Cloud and select New Agent to open the Editor.

The Start URL is where data collection begins, marking the starting point for your web data extraction agent.

For our example, we'll use the https://training.sequentum.com/ website.

Paste the URL below into the Sequentum Cloud Address Bar and press ‘Enter’ to load the page.

https://training.sequentum.com

This will load the target page in the Sequentum Cloud Agent Editor, enabling you to proceed with data extraction setup.

Selecting the Content to Capture

After loading the Start URL into Sequentum Cloud, you can select the data you want to capture and start building your agent. Move your mouse pointer over elements in the Agent Editor screen to see the items highlighted with a pinkish background color. This is how you can tell what will be selected when you click. (For a deeper dive, see the manual section on Selection Techniques)

To capture a list of all the elements present on the screen, click on the text “Build your first Agent” to select it. Then press and hold Shift and click on the next item in the list, “Multiple Elements”. This will select all the elements on the page that have the same XPath class. You will see that all the list elements are highlighted and the count of selected elements appears in the bottom left corner of the Editor window, as in the screenshot below.
Once all the elements are highlighted, look at the bottom frame as shown below and click on the Extract button. This will add a new List command along with Content command to capture the content. You will see these commands appear in the Agent Explorer pane on the left side of the screen.

Output Data Format

Now that you have your list selected, you need to define how you will export the data.

Sequentum Cloud supports exporting extracted data into many formats, including CSV and JSON. You can also insert data directly into databases such as Snowflake. Additionally, you can configure Sequentum Cloud to deliver data to remote storage options like S3, Google Sheets, Google Drive, and FTP.

To view the export options, find the Export icon in the toolbar on the far left, and click it once to expand the export options.
To choose your data export type, click on any of the Export method commands to add it to the Agent Explorer pane. (If you add one by accident, you can always remove an item from the Agent Explorer by dragging it out of the agent or by right-clicking it and selecting Delete from the right-click menu.)
CSV is selected by default and already appears in the Agent Explorer. For this sample agent, use CSV for the export method.

Testing the Agent

It's essential to test your agent to ensure correct data extraction. Sequentum Cloud offers debugging features to analyze agent performance and pinpoint issues.

To debug:

Click on the Play menu, then click on Play option.
Once you click Play, you will see the agent execute the commands in the Agent Editor frame. In this way you can monitor the agent as it goes through each command in the sequence. You can adjust the Play Speed to speed up or slow down the rate of the playthrough. You will see the steps of the playthrough in the Play Log in the bottom frame.
After the playthrough is completed, you can view the captured data from the bottom Data section or from the View>Play Data and confirm the output is correct or not.

Publishing the Agent

After completing your agent, publish the agent by clicking on the Publish option in the top menu of the Sequentum Cloud Editor. You will need to give your agent a Config name and select the Space where you want to publish it. Once Published, you can configure the agent’s schedule, shown in the next section.

Scheduling the Agent

Sequentum Cloud enables agent scheduling through the Control Center, allowing agents to run automatically at specified intervals.

Navigate to the Control Center by clicking on the Control Center option available on the top right corner on Sequentum Cloud Editor.
Clicking Control Center from within the Agent loads the Agent Details page:
Now click on Setup Run and enable the Schedule checkbox. Set a schedule and click on Save Task.
Note that Schedule is added and is visible under the Tasks section.
Here you will see the Export file and Log file and all the other details which you set previously.

Running the Agent

To run your agent:

Navigate to the Control Center and open the agent you created and published earlier.
Now click on ‘Setup Run’.
Click on the ‘Run Now’ option.
You will observe that the agent starts running.
After the agent gets completed, click on the option “View run details and files”.

Here you will get the Export file and Log file and all the other details previously set by the user.

Conclusion

Congratulations! You have created, scheduled, and run your first online agent!

Once you're comfortable creating an agent, you can continue reading the Sequentum Cloud Manual learn more about using Sequentum Cloud. You can also visit the Cloud Training site to practice additional techniques with the Sequentum Cloud Editor.