My feed runs painfully slow. What do I do?

The factor that affect speed of feed runs

There are several factors that affect the speed of feed runs. They are listed below, the most predominant first

1/ The number of products/variants. In general for a few thousands products, feed runs only take minutes to complete. Portfolio with tens of thousands products could take longer time, up to hours or even half a day.

2/ The nature of the feeds.

- Foreign feeds that contains metafields that require translations. Always use the option "Untranslated" for metafields that does not require translation (the same value across all languages). The app will then skip querying translations for these tokens which could save substantial amount of time

3/ Some types of feeds are slow or dependent on 3rd party queries. This include reviews feeds that query reviews from 3rd party app (Yotpo, Appio, etc.). Feeds that are translated by Weglot and GTranslate also require translations from these 3rd parties, which results in longer build time.

4/ The number of feeds. This actually does not affect the speed a lot, unless the extra feeds falls into the above categories (involve metafield translations or 3rd parties). Because of this the app is designed in a way that it rebuilds all the feeds even when changes were made to a subset of them. Think of making a meal for 1 person, and a group of 10. Cooking for 1 will take less, but most likely, not much.

Feed run queue

All the feed run requests, including manual ones (by pressing the manual refresh button or saving setting changes) and scheduled ones, are queued. The queue is consumed in batch of maximum 10 requests (yes, 10 tasks at a time, not one by one).

Think of the queue consumption like a cargo train: if manual refresh button was clicked 15 times, the next pickup would pick first 10 request, and run the latest one of the 10. After the run finish, it take the other 5 (if there are no more added to queue during the run of the first 10) and run the latest of the 5, and so on.

Of course, if there are less than 10 requests in the "cargo", the train will still run.

An interesting point to note is that since feed runs evaluate products in sequence, changes made to the products can be picked up in the same run, as long as they were made before the time the run reach those products.

If your feeds take 10 hours to complete after a feed settings change, it may sounds unbearable at first. However it may not be as a big problem as it seems, because eventually we will reach a point where feeds are optimized and changes are no longer needed too frequently. What is more important is that changes will be picked up at least once a day, which is sufficient for most ads platform (Google Shopping only allow file feeds to be fetch most frequently once a day).

Consistency (and transparency of the feed update times) is where our app shines compared to our competitors.

For Google Shopping, in most cases, as long as there is no price or availability mismatch errors, the listing is fine. We do have solution for edge cases (by creating API supplement feeds alongside with file feeds for extremely large portfolio Which feed type should I choose, API feeds or file (XML, CSV, TSV ) feeds?).