Performance Tree performance-tree

Scope scope

The following diagram is meant to provide guidance on the steps to take to troubleshoot performance issues. It is split in five sections for easier reading.

Each step in the diagram is linked to a documentation resource or a recommendation.

Prerequisites and Assumptions prerequisites-and-assumptions

The assumption is that a performance issue is observed on a given page (either an AEM console or a web page) and can be reproduced consistently. Having a way to test or monitor the performance is a pre-requisite before starting the investigation.

The analysis starts at step 0. The goal is to determine which entity (Dispatcher, external host, or AEM) is responsible for the performance issue then determine which area (server or network) should be investigated.

Section 1 section

chlimage_1-103

Section 2 section-1

chlimage_1-104

Section 3 section-2

chlimage_1-105

Section 4 section-3

chlimage_1-106

Section 5 section-4

chlimage_1-107

Step
Title
Resources
Step 0
Analyze Request Flow

You can use standard HTTP request analysis in the browser to analyze the request flow. For more info on how to do this analysis on Chrome, see:

https://developer.chrome.com/docs/devtools/

Step 2
Are requests coming from external hosts?
You can use standard HTTP request analysis in the browser to analyze the request flow. See the above links on how to do this analysis on Chrome.
Step 3
Can the requests be cached?
For more information on cacheable requests and general Dispatcher performance optimization advice, see Dispatcher Performance Optimization.
Step 4
Are requests coming from the Dispatcher?
To see if the requests are cached properly, check the Dispatcher debugging documentation.
Step 5
Is the Dispatcher trying to authenticate each request via AEM?
Check if the Dispatcher sends HEAD requests to AEM for authentication before delivering the cached resource. Look for HEAD requests in the AEM access.log. For more information, see Logging.
Step 6
Is the geographic location of the Dispatcher far away from the users?
Move the Dispatcher closer to the users.
Step 7
Is the network layer of the Dispatcher OK?
Investigate the network layer for saturation and latency issues.
Step 8
Is the slowness reproducible with a local instance?
Use Tough Day to replicate "real-world" conditions from the production instances. If this scenario is not realistic for the space of your development, make sure to test the production instance (or an identical staging one) in a different network context.
Step 9
Is the geographical location of the server far away from the users?
Move the server closer to the users.
Steps 10 and 29
Investigate network layer

Investigate the network layer for saturation and latency issues.

For the author tier, it is recommended that the latency does not surpass 100 milliseconds.

For more information on performance optimization tips, see this page.

Step 11
Move server closer or add one per region
Step 12
Troubleshoot AEM server
Check the following sub steps in the diagram for more information.
Step 13
Check hardware requirements
Check the documentation on Hardware Sizing Guidelines.
Step 14
Check for frequent causes of performance issues
Step 15
Find slow requests

You can check for slow requests by analyzing the request.log or by using rlog.jar.

For more information about using rlog.jar, see this page.

See Find requests with long duration times using rlog.jar.

Step 16
Profile server
For information about profiling tools you can use with AEM, see Tools for Monitoring and Analyzing Performance.
Step 17
Find slow methods in profiling
Step 18
Common scenarios of profiling
See Analyzing Specific Scenarios in the Performance Optimization section.
Step 19
100% CPU
https://experienceleague.adobe.com/docs/experience-manager-release-information/aem-release-updates/previous-updates/aem-previous-versions.html
Step 20
Out of memory
  1. Out Of Memory
  2. My application throws out-of-memory errors
  3. Analyze Memory Problems.
Step 21
Disk I/O
See the Disk I/O section in the Monitoring and Maintaining documentation.
Steps 22 and 22.1
Cache ratio
See Calculating the Dispatcher Cache Ratio.
Step 23
Slow queries
Best Practices for Queries and Indexing
Step 24
Repository tuning
Step 25
Workflows running
Step 26
MSM Infrastructure
Multisite Manager Best Practices
Step 27
Assets tuning
  1. Assets Synchronization Service
  2. Multiple DAM Instances
  3. Performance tuning tips article here.
Step 28
Unclosed sessions
Checking for unclosed JCR sessions
Step 30
Move Dispatcher closer (add one per "region"?)
Step 31
Use CDN in front of Dispatcher
Using Dispatcher with a CDN
Step 32
To offload the AEM server, use session management at the Dispatcher level
Enabling Secure Sessions
Step 33
Make requests cacheable
  1. General Dispatcher Configuration
  2. Configuring the Dispatcher Cache

How to improve cache ratio; make requests cache-able (Dispatcher best practices)

Also, consider the below settings to optimize your caching configurations

  1. Set a no-cache rule for HTTP request that is not GET
  2. Configure query strings to not be cacheable
  3. Do not cache URLs with missing extensions
  4. Cache authentication headers (possible since Dispatcher version 4.1.10)
Step 34
Upgrade Dispatcher version

You can download the latest Dispatcher version at this location:

Follow link

Step 35
Configure Dispatcher
Configuring the Dispatcher
Step 36
Check cache invalidation
Steps 37 and 38
Lazy-loading
See the Gem Session on AEM Web Performance.
Step 39
Use pre-connect to reduce connection overhead
See the Gem Session above. Also, additional pre-connect documentation on W3c: https://html.spec.whatwg.org/#linkTypes
Steps 40 and 41
External hosts latency and response time
Investigate the latency and response time for the external hosts.
Steps 45
and 47
Using HTTP/2
See the Gem Session for steps 37,38 and 39. Also, check out this forum post on HTTP/2 support.
Step 49
Shrink payload size
Enable Gzip and shrink the image size.
Steps 42 and 43
Keep-Alive

Is the Keep-Alive header present in the different requests to reuse connections? Otherwise, it would mean that each request leads to another connection establishment, which introduces unnecessary overhead. (Standard HTTP request analysis in the browser)

You can check the Proxy Server tool to check for Keep-Alive connections.

Step 44
How many requests are made?
Perform standard HTTP request analysis in the browser.
Step 46
Reduce number of requests
  1. Concatenate resources (images, CSS sprites, JSON)
  2. Clientlibs embedding:
    1. Creating Client Library Folders - see heading Using embedding to minimize requests
Step 48
What is the size of the payload?
Standard HTTP request analysis in the browser
Steps 50 and 51
JS code blocking
https://experienceleague.adobe.com/docs/experience-manager-gems-events/gems/gems2016/aem-web-performance.html
recommendation-more-help
19ffd973-7af2-44d0-84b5-d547b0dffee2