Different integration Task states

A Task is generated whenever data moves through Alumio, and it helps track data at every step of the integration process. It provides insights into the status of the integration, whether it's successful, in progress, in queue, if it’s being skipped, or if it has encountered errors. Thus, apart from measuring data exchange within Alumio, Tasks also act as the foundation for monitoring and troubleshooting integrations.

This is why, one of the first things that the Alumio dashboard shows you is the real-time status of your Integration Tasks. Apart from showing you the total number of Tasks you’ve generated, the Task monitor on the Alumio dashboard gives a clear overview of the different stages in the lifecycle of a Task within Alumio. It breaks down the journey of data moving from one application to another as follows:

• New Task: When data is first imported from a source as part of a Route, it’s recognized as a new Task.
• Processing Task: Once Alumio begins sending this data to a target app, the Task moves into processing.
• Finished Task: If the task is completed successfully, it is marked as finished.
• Failed Task: If an error occurs during the data exchange, the task is labeled as failed.
• Skipped Task: In scenarios such as testing or when certain data needs to be filtered, Tasks can be deliberately skipped manually or in an automated way.

Let’s briefly explore how each of these Task statuses are vital to how the Alumio iPaaS helps streamline and monitor the entire integration process every step of the way.

New Task:

To integrate data between applications, you have to first gather data from a source application (System A) and then send that data to a target application (System B). A new Task is generated when Alumio successfully imports data from System A, via an Incoming configuration, within a Route.

When setting up this Incoming Configuration, Alumio allows you to define exactly what data to retrieve and how often to retrieve it. Do you need to pull in all available data or only specific parts of a data entity (e.g., Orders from a specific region)? Should the data format be converted (e.g., from XML to JSON)? How frequently should the data be imported—every minute, hour, day, or week? Alumio provides Transformers to modify and structure the incoming data and a Scheduler to determine when and how often the data should be retrieved. This ensures that you can be very specific about what kind of data you wish to send to the target application.

For example, when integrating an ERP system with both an e-commerce platform and a PIM system, Alumio retrieves product data and creates a New Task, queuing it for processing. At this stage, you can filter specific records (e.g., only updated products), standardize formats (e.g., convert currencies), and enrich data (e.g., add missing attributes). This allows you to first collect and structure the data within Alumio before deciding how to transform it for different destinations. You might map product categories differently for the PIM system while adjusting pricing formats for the e-commerce platform—all from the same New Task.

Note: If an incoming configuration is not configured within an (active) Route, a new Task will not be created.

Processing Tasks:

When you create an Outgoing Configuration and run the Route in Alumio to send data that you’ve pulled in from a source (system A) to a target application (system B), a New Task turns into a Processing Task.

As indicated earlier, this is the reason Alumio doesn’t immediately send (export) data that it pulls in (import) via an Incoming Configuration and queues it as a New Task. It is so you can choose where and how you wish to send this data by building an Outgoing Configuration. Do you need to filter, modify, or map the format of the data you’ve imported to match the target application? Or, do you want to send the data in batches every minute, hour, week, etc.? Alumio allows you to decide how you want to Transform and Schedule this data that you’ve pulled in from System A before sending it out to system B.

As per our earlier example, scheduling the sending of data that you’ve pulled in from the ERP system to the e-commerce platform creates a Processing Task. A Processing Task goes into the queue within Alumio and gets ready to send the data when scheduled.  
‍

Note: Alumio also allows users to select the “Real-time processing” option when building a Route, in the case where critical data needs to be integrated immediately. In this case, the moment the Route is run, a Processing Task is directly created instead of a New Task, and it immediately seeks to become a Finished Task.
‍

Finished Tasks:

A Processing Task becomes a Finished Task when the data exchange between the source and target application is successfully completed. This means that Alumio has successfully sent the imported data to the target system, and the transformation or scheduling rules you configured have been applied without any issues.

For example, if the ERP system successfully sends product updates to the e-commerce platform, and all the data has been received, processed, and stored correctly, the Task moves from Processing to Finished status.

Failed Tasks:

A Processing Task becomes a Failed Task if something prevents the data exchange from being processed successfully. This could be due to a range of issues, such as:

• The target application (System B) being temporarily unavailable.
• The data format not matching the expected structure in the receiving system.
• Authentication or permission errors preventing data exchange.

When a Task fails, Alumio provides error logs and troubleshooting options to help you quickly identify and resolve the problem. You can retry the Task once the issue is fixed, ensuring that no data is lost.

For instance, if an ERP tries to send product updates to an e-commerce system, but the SKU field is missing or incorrectly formatted, the Task will fail, and the error details will be logged for review. You can then check the logs to identify and swiftly resolve the error and retry the Task.
‍

Skipped Tasks:

There’s also a “Skipped Tasks” in the Alumio iPaaS that counts those Tasks that, for various reasons such as testing or predefined filtering rules, are intentionally bypassed. A Task is categorized as “Skipped” when Alumio detects the data being processed doesn’t need to be sent to the target system. This happens when:

• The data already exists in the target system and hasn’t changed.
• The integration logic has conditions that determine whether data should be processed (e.g., only sending new orders but skipping already processed ones).
• The transformation rules filter out specific data before it reaches the outgoing configuration.

Alumio allows you to manually skip specific data or apply rules to automatically skip specific data in bulk. For example, let’s say that an ERP is set to send product updates to an e-commerce platform every 10 minutes, but no changes have been made to the product data since the last update. Alumio can recognize this and mark the Task as Skipped. This will prevent unnecessary processing. This helps optimize performance and avoid redundant API calls.

Since skipped data remains stored within Alumio, it also enables the Alumio iPaaS to act as a caching system. This means that if previously skipped data becomes relevant later, such as an order moving from ‘Pending’ to ‘Paid,’ you can easily reprocess it without needing to pull it again from the source application. This improves efficiency and reduces the load on connected systems.

‍

How Alumio can be used as a caching system

Alumio can also be used as a caching system, wherein only delta versions of data, or only updated or new versions of data, will be sent from one application to the other.

For instance, let’s say we regularly pull in a thousand records, which amounts to thousands of products, from one application to the other. It’s productive to send data for only those products that have an updated price, stock, inventory, etc., to the other application, or data for products that are entirely new, and haven't been registered in the other application yet.

With this approach, you can schedule an update every minute, sending only a handful of Tasks — such as three, five, or, at peak times, maybe twenty new product updates. This is much more efficient than trying to send all one thousand records every minute. Checking regularly for new data and sending only this data ensures that we're not wasting time and resources in processing redundant Tasks. If you need to see the data that’s filtered out, you can check the Logs that Alumio helps maintain for each action and integration.
‍

Other essential Task statuses within Alumio

Beyond the primary Task statuses displayed on the dashboard, Alumio also manages additional Task statuses that play a crucial role in ensuring smooth data processing. These statuses are “Retry”, “Rejected”, and “Waiting”, which help automate error handling, enforce data validation, and provide manual control over specific Tasks when needed.
‍

Retry Tasks:

It’s possible that you may need to reprocess a Task to export the data again to the target system. This could be because the Task previously failed due to a configuration issue that has since been fixed, or simply because all entities need to be reprocessed—even if they were already marked as Finished.

When the “Activate retrying of failed Tasks” option is enabled in a Route, Alumio will automatically attempt to reprocess Failed Tasks. This means Alumio will automatically attempt to reprocess the Task until it either succeeds or reaches the configured retry limit. This feature helps recover from temporary issues, such as brief system outages or network disruptions, without requiring manual intervention. If a Task continues to fail after exhausting the maximum retry attempts, it will be marked as a Failed Task, and the error details will be logged for troubleshooting.

Retrying Tasks manually will set the status of the Tasks to New again, so the route will process it to the outgoing system in the next run. On the Task overview page, you can filter and choose to Retry several Tasks at once. You can also choose to select a single Task and Retry it. This way, you can only retry a task that has the status Finished, Failed, or Skipped.
‍

Rejected Tasks:

A Task is categorized as Rejected when it fails to meet specific system constraints or validation rules. For instance, if a limit of 1000 bytes is set for entity data, and a Task exceeds this threshold, it will be rejected instead of being processed. Similarly, Tasks with invalid entity data may be automatically rejected to prevent errors from propagating through the integration. The reason for rejection is recorded in the Import Logs on the Task details page, allowing users to review and address the issue before resubmitting the data.

‍

Waiting Tasks:

In some cases, users may configure a Route so that instead of a Task automatically being marked as Finished or Failed after processing, it enters the “Waiting” status. This provides users with an opportunity to manually review the Task and determine its outcome based on the information available in the Logs. If a Task is in the “Waiting” status, users can decide whether it should be marked as Finished or Failed by using the “Actions” menu. Bulk actions can also be applied to update multiple Waiting Tasks at once. This status is particularly useful for cases where manual validation is required before finalizing the integration process.

Now that we’ve understood the different types of Tasks within Alumio, let’s explore all the details that a Task contains to understand how it, furthermore, helps track, manage, and streamline integration at every step of the way.
‍