ProductTileWarmup

Owned by Marta Siewierska-Stańczak
Last updated: Apr 21, 2023 by Dariusz Matoga
5 min read

https://github.com/magesuite/product-tile-warmup

This module was created in order to efficiently warm up product tiles. It creates a separate page that is able to display only tiles that are not yet in the Redis cache.

This feature periodically and intelligently crawls special product tile pages in order to pre-generate the related internal Magento caches thus considerably speeding up page load times for the customers.

 

Installation

This module is part of MageSuite metapackage

 

Installation if metapackage is not used:

composer require "creativestyle/magesuite-product-tile-warmup" ^1.0.0

Admin settings

Settings are located in Stores -> Configuration -> MageSuite -> Product Tile Warmup

 

Basic settings:

Field

Options

Comment

Field

Options

Comment

Tile Crawling

Enable Disable 

 

Enable or disable the product tile warmer crawling mechanism.

Localhost mode

Enable/Disable 

When it is enabled then requests are sent to 127.0.0.1 instead of Base URL.

Debug mode

Enable/Disable 

Debug mode will log all actions performed by tile warmup workers into var/log/tile_warmup.log file.

Cron schedule

 

Provide cron schedule in * * * * * format

Customer Groups

(list of all customer groups from store)

 

Customers to warm tiles up for.

A special customer account will be created in each of these groups (except the guest group) and the tiles will be warmed up while being logged in as each one of them. Enable only the most frequently used customer groups to minimize the performance impact of the crawler.

Disabled Store Views

(list of all customer stores)

 

By default all store views are enabled. Disable only store views that do not display any products.

Workers processes

 

If you want multiple stores to be warmed up in parallel, set them to use separate processes.

 

Basic auth settings

If the shop frontend is protected by the basic auth, please provide the credentials here.

Field

Option

Coment

Field

Option

Coment

Username

 

 

Password

 

 

 

Backend

Backend moved from internal space. To be reviewed.

General concept

In order to efficiently warm up product tiles, we created a separate page that is able to display only tiles that are not yet in the Redis cache.

The whole warm-up process is based on simple PHP CLI processes called workers that refresh the above page as frequently as possible while trying to not overload servers.

When the worker detects that it is no longer warming up any tiles (= all tiles got warmed up) it will delay time between future requests to decrease the server load.

Such delay can get reset when:

  • BLOCK_HTML cache type gets cleared (this includes full cache flush scenario)

  • any product-specific cache tag is cleared (e.g. cat_p_10)

When a reset happens, the worker tries refreshing the tile warmup page immediately to warm cleared tiles as fast as possible.

Configuration

In order to optimize the whole process there are three main configurations that can be done in admin panel:

After changing configuration here, workers needs to be reinitialized.
For this to happen developer needs to kill all workers processes on the server. They will be automatically reinitialized by cron depending on Cron Schedule setting (or you can use magerun to execute initialize_tile_warmup_worker cron job right away)

Which customer groups should get warmed up

By default, tiles get warmed-up for guest and general customer groups (default one for logged-in customers)

Which store views should not get crawled

By default, all store views get warmed up and only excluded store views can be defined.

This ensures that when we create a new website/store view, it gets automatically warmed up without a need to make additional configurations.

Workers grouping

It is possible to warm up tiles for multiple store views in parallel. This is very useful in case we have some popular store views that should always get warm up as fast as possible and should not get held up by warmup done for less popular store views. LEDs project is a good example where German should always be warmed up first, and English/French stores can wait with their warmup.

To do so, every popular store view should get Run in a separate process group set to Yes and a unique numerical Group Id assigned.

This will cause that specific store view to be warmed up by a completely separate PHP worker process.

By design, workers are implemented in plain PHP and do not bootstrap Magento in order to consume as small as it is possible amount of RAM memory.

Running multiple workers should not cause even small servers to run out of memory.

Basic auth

In case the shop is protected by a basic auth. Please provide a basic auth username and password in Basic auth settings group.

This is not done by default by a deployment process.

Testing if the feature works correctly

There is no easy way of making automatic tests for this feature since it requires Redis to be configured and that is not available when integration tests are run.

The only manual way of testing this feature is to:

  1. clear block_html cache in Cache Management in admin panel

  2. enter tile/warmup page for a specific store, for example for de store on our test system: https://magesuite-test.magesuite.net/de/tile/warmup

The page should return different products on every refresh and eventually should display no products and x-already-warmed-tiles-count header should return a number greater than 0 and similar to the number of visible products in the shop.

The above test can be executed for different store views and customer groups as well. To check customer group you need to log in to an account that has desired customer group assigned.

To test if workers are initialized and working correctly enable Debug mode in the admin panel. Kill all existing workers processes. Wait for cron to reinitialize them, and observe contents of var/log/tile_warmup_debug.log file.

Frontend

No frontend functionalities in the module.

 

 

Â