If you’re running a larger WordPress site, you may be facing performance challenges that can impact user experience and search engine rankings. Fortunately, there are several steps you can take to improve your site’s speed and performance. This article covers some of the most common performance challenges for larger WordPress sites and provides tips and solutions for addressing these.
Growing pains… some should be so lucky to have them!
Let’s talk about how to manage a WordPress site that is growing too quickly too soon and causing you or your clients all sorts of pains and problems.
If the issue is temporary, such as dealing with an unexpected traffic spike from a post gone viral (another thing we should be so lucky to experience!), then knowing how to scale your WordPress site when traffic soars can fix this.
However, if problems persist, it may take more than a couple of aspirins and calling the doctor in the morning to make the headaches go away.
In this article, we’ll cover:
WordPress Enterprise Development Challenges
Q: How complex can you make a WordPress site?
A: Very.
When it comes to building large and complex sites, WordPress’s capacity to handle it is not an issue. As WordPress enterprise developer and global SME business adviser Mario Peshev states in his excellent article on building large and complex sites using WordPress…
“WordPress is a proven CMS that handles various applications handling millions of users and tens or even 100M views a month.”
As Mario also states…
“Scaling from 10M to 50M is feasible, 50M to 100M is challenging, 100M–200M is quite complex and 200M+ may require some serious engineering effort.”
So, the capacity of the WordPress CMS platform to handle large and complex sites is not a problem.
The issue is having the skills to handle WordPress enterprise development challenges.
As most developers know, WordPress is not only a widely popular content management system known for its flexibility, ease of use, and affordability, but it is also an excellent platform for small businesses and startups that want to establish a web presence quickly and easily.
However, when it comes to enterprise-grade WordPress development, the amount of information available is as scarce as a developer who hasn’t resorted to cursing loudly at their code editor at least once.
So, before we get into diagnosing the challenges and issues of dealing with large WordPress sites, let’s explore some of the challenges of finding relevant information on WordPress enterprise development.
Here is a summary of the points Mario Peshev makes in his article…
Scarcity of Information on Enterprise-Grade WordPress Development
One of the main reasons why information on enterprise-grade WordPress development is scarce is that only a handful of agencies specialize in building WordPress platforms, applications, plugins, or performing migrations and integrations for the enterprise.
Most vendors profile in small business websites, and only a small chunk of the service providers work with enterprises.
Furthermore, those consultants and agencies often don’t have the time and resources to write tutorials and share their know-how with the industry, or they just don’t care, especially more hardcore engineers who don’t want to bother.
Another reason why information on WordPress enterprise development is limited is that WordPress is often not the core application that enterprises use in the first place. It’s another obstacle for many, like working on the front-end interface as 1% of the main platform running behind the scenes.
However, WordPress developers who want to bid on enterprise projects can focus on several different areas to enhance their expertise.
Focus on Different Areas for Enhancing Expertise
The first area that WordPress developers should focus on is studying the WordPress Core, APIs, and the surrounding ecosystem in-depth. This will give developers a deeper understanding of the platform and how it works.
They should also make sure that they’re comfortable with WordPress coding standards and best practices. This will ensure that the code they write is maintainable and easy to read.
The second area that WordPress developers should focus on is practicing in the main technical areas that enterprises care about, such as performance, security, scalability, and backward compatibility.
Enterprises have high expectations, and it’s essential to demonstrate that you have the expertise to meet their requirements.
These WordPress development resources will help you gain these valuable skills and expertise:
- From WordPress Beginner to WordPress Pro: 200+ Career-Boosting Resources
- Courses that Turn WordPress Users into Developers
Strategic Players in the Field
Hosting vendors are strategic players in the field and occasionally work with high-scale applications. Developers can browse their resources and follow their blogs, knowledge base articles, and the like. WordPress is a platform built on top of PHP and SQL, front-end served through HTML, CSS, JavaScript. It runs on a web server like Apache or Nginx using mod_php or php-fpm, connected to a MySQL database on a Linux server.
Most of the heavy lifting for enterprises happens on top of those layers. Therefore, it makes sense to dive deeper into their own communities and resources stressing on those topics.
Follow WordPress Core Contributors and Employees
It always helps to follow WordPress Core contributors, employees at enterprise-grade companies, and the blogs of the leading agencies working with enterprises. You may find some relevant case studies, interviews with clients, or other top engineers that could help you improve even further.
Now that we’ve looked at the first challenges, which is acquiring the expertise to handle large and complex WordPress sites and meeting the expectations of enterprises, let’s turn to addressing common performance issues you may experience working with large WordPress sites.
Common Performance Challenges for Large WordPress Sites
WordPress is used by some of the biggest and most well-known companies, celebrities, and brands in the world, like Intel, Pepsi Cola, PlayStation, American Express, TechCrunch, Fisher-Price, Beyonce, Justin Timberlake, Usain Bolt, and many more.
Someone has to look after these large sites… why not you?
While browsing through WPMU DEV’s member forums (which, by the way, is a treasure trove of information for web developers), I came across this post from WPMU DEV member Charly Leetham, which I am reproducing in full below:
***
I was contacted by a long term client asking for assistance with their client.
The end customer is setting up a rather large website in WordPress and they were having no end of difficulties in keeping the site running. It was so bad, that they had to reboot their Amazon EC2 instance regularly (several times a day regularly).
With trepidation I agreed to take a look and see if I could help. What I found has left me … saddened. For the client, mostly.
The site:
- Database: 4Gigabytes (after optimization)
- Posts / Pages and other content: Over 900K entries.
This is not a small site.
It was built in Elementor which initially left me concerned, as I know that Elementor is resource hungry.
The EC2 instance was provisioned with 140 Gig storage and 32 Gig memory. More than enough, right? One would think so.
The business had been moved to EC2 by a consultant who had promised them it would improve their performance. Then they told them that the reason the instance kept hanging was because of the high number of transients that were being created.
They created a cron job that deleted the transients every hour and with very little improvement.
I’ve found a number of things during my investigations but the three most concerning things are:
1. Although the server was provisioned with 32G of memory. PHP had been limited 2G and WordPress had been limited to 40M.
It’s no wonder they were having trouble.
Increasing these limits has stopped the hanging but we’re still experiencing memory overflows.
2. The database was provisioned on the same server.
Splitting the database onto a RDS (remote database server) should provide more performance increases.
3. No optimization or performance improvement work had been done.
By implementing Hummingbird, I’ve been able to improve the load time of the site and that’s without doing anything really hard core. That’s still to come.
The main thing I want to highlight for others here, is that it’s the incremental knowledge you bring to the table when working with clients.
Yes, people can build their own WordPress sites but few people can really make them hum. That takes experience and a lot of work.
***
Charly’s forum post is a great example of some of the typical performance challenges you can expect when working with larger WordPress sites and provides a number of useful insights into handling these.
To address these challenges, let’s first summarize the main technical issues Charly described when looking at this client’s site:
- The end customer is setting up a rather large website in WordPress with over 900k entries and a 4GB database after optimization, which is not a small site.
- The website was built in Elementor, which is resource-hungry and requires a lot of server resources.
- The EC2 instance was provisioned with 140GB storage and 32GB memory, but PHP had been limited to 2GB and WordPress had been limited to 40MB, causing performance issues and memory overflows.
- The database was provisioned on the same server, which caused performance issues. Splitting it onto a remote database server should provide performance improvements.
- No optimization or performance improvement work had been done. By implementing Hummingbird, Charly was able to improve the site’s load time.
- The incremental knowledge and experience brought to the table by an experienced web developer is crucial for optimizing and improving the performance of WordPress sites, which can be complex and require a lot of work to make them run smoothly.
We’ve already addressed point #6, so let’s go through the other issues on the list above.
Large WordPress Site Performance Issue #1 – WordPress Database
As your WordPress site grows, so does the size of its database. Your WordPress database can become quite large and may start causing some issues.
Managing a large WordPress database can be a daunting task, so let’s take a look at some of the challenges, best practices, strategies, and solutions for managing your WordPress database on larger sites.
The challenges of having a large WordPress database include:
- Slow page load times: A large database can slow down your website, making it difficult for visitors to load pages quickly.
- Backup and restore issues: Backing up and restoring a large database can be a challenge, and it may take a long time to complete the process.
- Database corruption: A large database can be more prone to corruption, which can cause data loss and other issues.
- Difficulty in database maintenance: Maintaining a large database may require more resources and expertise to keep it running smoothly.
Here are some strategies and best practices managing WordPress databases on larger sites:
Initial Configuration
Before you even start thinking about managing your database, it’s important to make sure that it’s set up correctly. When you install WordPress, it creates a new database for you. However, if you’re running a large site, you may want to consider using a separate database server. This will help to improve performance and reduce the load on your web server.
When configuring your database, it’s important to choose the right settings. In particular, you’ll want to pay attention to the database character set and collation. These settings can affect how your content is displayed on your site, so it’s important to get them right from the start.
Where to Keep the Databases
When managing a large WordPress site, you’ll want to think carefully about where to keep your databases.
There are a few different options to consider:
- Local Database: You can keep your database on the same server as your website. This is the simplest and most common option, but it can lead to performance issues as your site grows, as Charly referred to in the client example above.
- Remote Database: You can keep your database on a separate server, either within your own network or in the cloud. This can improve performance, but it can also increase costs.
- Managed Database: You can use a managed database service, such as Amazon RDS or Google Cloud SQL. This can be a good option if you don’t have the expertise to manage your own database.
Database Access Time with Large Numbers of Records
As your WordPress site grows, the size of your database can have an impact on how quickly your site loads.
When you have a large number of records in your database, queries can take longer to run, which can slow down your site.
Caching can help speed up your website by storing frequently accessed data in memory, reducing the need to access the site’s database and PHP. This, of course, depends on the kind of caching being used, e.g. database caching (which includes object caching) or page caching (where the cache of the web pages is stored and presented when a specific page is requested later without needing to be processed by PHP and MySQL).
To improve performance, you can use server-side caching, caching plugins that manage server-side caching solutions, or standalone caching plugins. Our performance-optimizing plugin Hummingbird, for example, has its own caching but also integrates with WPMU DEV’s server-side caching.
Caching can have a significant impact on site performance, particularly for larger sites. However, setting up and managing caching can be complex and time-consuming.
Also, it’s important to regularly monitor your site’s performance to ensure the caching is optimized for your specific needs.
To learn more about caching solutions, check out our Ultimate Guide to WordPress Caching.
Another option is to use a technique called “sharding,” which involves splitting your database into smaller pieces. This can help to improve performance by spreading the load across multiple servers.
Techniques for Splitting the Data Up
If you’re using a technique like sharding, you’ll need to decide how to split your data up. One option is to split your data by category or tag. For example, you could have one database for posts related to technology, and another for posts related to entertainment.
Another option is to split your data by date. This can be particularly useful if you have a lot of older content that doesn’t change very often. You could have one database for posts from the last year, and another for older posts.
Consider also using a plugin like HyperDB. HyperDB is maintained by Automattic, the parent company of WordPress.
As described on the plugin page…
HyperDB allows tables to be placed in arbitrary databases. It can use callbacks you write to compute the appropriate database for a given query. Thus you can partition your site’s data according to your own scheme and configure HyperDB accordingly.
Basic Indexing
Indexing your database can help to improve performance by making it faster to search for data. When you create an index, the database creates a data structure that makes it easier to search for specific values.
To create an index, you’ll need to use the MySQL command line or a tool like phpMyAdmin.
When you’re creating an index, it’s important to choose the right columns to index. Typically, you’ll want to index columns that are frequently used in queries.
You can also use a plugin like Index WP MySQL for Speed. This plugin adds database keys (also called indexes) to your MySQL tables to make it easier for WordPress to find the information it needs. The plugin page also includes excellent information on database indexing in relational database management systems.
Settings and Logs to Check
To keep your database running smoothly, there are a few settings and logs that you’ll want to keep an eye on. These include:
- MySQL slow query log: This log records queries that take longer than a certain amount of time to run. By analyzing this log, you can identify queries that are causing performance issues.
- MySQL error log: This log records any errors that occur in the MySQL server. By monitoring this log, you can identify and troubleshoot issues that may be affecting your database.
- WordPress debug log: This log records any errors or warnings that occur within WordPress. By monitoring this log, you can identify issues with your WordPress installation or plugins.
- Database backups: Regularly backing up your database is important to ensure that you don’t lose any data in case of a server crash or other disaster and can restore your website quickly in case of a problem. You can use a plugin like Snapshot to automate this process, or if you’re hosting with WPMU DEV, you can configure automatic enterprise database backups to perform daily and even hourly. Also, consider storing all backups separately from the server hosting the site, as the backups may be lost if the server crashes.
Other Ongoing Maintenance
In addition to the above, there are a few other ongoing maintenance tasks that you’ll want to perform to keep your database running smoothly.
These include:
- Cleaning up your database: Over time, your database can become cluttered with unused data. Check our article on how to clean up your database and remove unnecessary data for more details.
- Optimizing your database tables: Reducing the size of your database and optimizing your database tables helps to improve site performance. You can optimize your database by removing unnecessary data, such as post revisions, trashed items, spam comments, and unused plugins and themes. Check our complete WordPress database optimization guide for detailed instructions and plugins that help you do this.
- Monitoring your site for security issues: Large sites are often a target for hackers. You can use a plugin like Defender to monitor your site for security issues and prevent attacks.
In terms of cleaning up your database, Charly mentions a high number of transients as being a possible issue affecting the site’s performance. Although addressing this issue seemed to offer very little improvement in Charly’s client’s case, it’s worth mentioning it here as something to check if you are experiencing issues with your site.
Transients are a type of cache that stores data in the database for a specific period of time. They are used to speed up the loading time of a website by storing the results of a complex or time-consuming query, such as an API request, so that the query doesn’t have to be run every time the page is loaded.
Transients have a set expiration time, after which they are automatically deleted from the database. However, if the website is not properly optimized, transients can accumulate in the database and cause performance issues, such as slow page loading times or database crashes.
To optimize WordPress and avoid issues with transients, there are several steps that can be taken. These include:
- Use a caching plugin: A caching plugin like Hummingbird can help reduce the number of database queries and prevent unnecessary creation of transients.
- Delete expired transients: Expired transients can accumulate in the database, so it’s important to regularly delete them to keep the database optimized. This can be done manually, or by using a plugin like Hummingbird.
- Set a maximum lifetime for transients: By setting a maximum lifetime for transients, you can prevent them from being stored in the database for too long, which can lead to performance issues. This can be done using the set_transient() function in WordPress.
- Use a remote database: Storing the database on a remote server can help reduce the load on the server and prevent issues with transients.
- Increase the memory limit: Increasing the memory limit for PHP and WordPress can help prevent memory overflows and performance issues caused by transients.
No matter what size WordPress site you are working on, using WPMU DEV’s Hummingbird caching and site optimization plugin can help to automatically take care of expired transients and eliminate this issue, leading to faster page loading times and a smoother user experience.
In terms of increasing the memory limit for PHP, if you are a WPMU DEV member, it’s really easy to check a whole bunch of information about your WordPress site, include current PHP memory limits and max filesize upload settings.
Just log into your WordPress dashboard and navigate to the WPMU DEV dashboard plugin menu. Select Support > System Information > PHP tab.
If you are not a WPMU DEV member, you can still check this information manually.
To find out how much php memory is allocated, create a php and add the following:
Call it something like php-test.php and upload it to your server.
Access the file from a browser and search for memory_limit. This will give you two settings – what the local site settings are and what the server default is. It is possible to have different php memory_limits by site.
For WordPress memory, for instance, you might see the following:
define('WP_MEMORY_LIMIT', '64M');
Note that if this entry is missing in the wp-config.php file, then your site is probably working between 40M and 64M.
In addition to the above, make sure to also scan and fix corrupt or broken files and database in WordPress.
As you can see, there are quite a number of things you can do to improve the performance of your WordPress database.
Let’s move on, to…
Large WordPress Site Performance Issue #2 – WordPress Core, Themes, And Plugins
Charly mentions that another possible reason for the performance issues her client’s site was experiencing was using a resource-hungry theme.
Rather than focusing on a particular theme, let’s look at themes and plugins in general (btw… if you use Elementor, check out our article on how to optimize Elementor themes. We’ve also written articles on ways to optimize themes like Divi, WPBakery, Astra, and other page builders.)
Here are some of the things you can do:
Theme and Plugin Bloat – Themes and plugins can significantly impact the performance of a WordPress site, particularly if they are not optimized or updated regularly. Some themes and plugins can also be poorly coded, leading to slow loading times and site bloat.
Solution: Be sure to choose a lightweight and optimized theme that is regularly updated by the developer. Avoid using too many plugins and remove any unnecessary ones to reduce site bloat. Always keep your themes and plugins up-to-date to ensure optimal performance.
- Avoid poorly coded themes and plugins, as these can lead to slow loading times, site bloat, and conflicts.
- Choose lightweight and optimized themes and plugins that are regularly updated by their developer.
- Check your server logs to identify heavy plugins and themes that could be slowing down your site.
- Always keep your themes and plugins up-to-date to ensure optimal performance.
- Deactivate and remove unnecessary and non-essential plugins and themes.
As with all WordPress sites, regardless of size, it’s also really important to optimize your client sites.
There are a number of tools you can use to scan your site and measure site performance, including Google PageSpeed Insights and GTmetrix. These tools provide important insights into ways to optimize your sites.
You can also use a developer tool plugin like Query Monitor to help you identify issues, aid in the debugging of database queries, PHP errors, hooks and actions, block editor blocks, enqueued scripts and stylesheets, and HTTP API calls. The plugin also provides advanced features such as debugging of Ajax calls, REST API calls, and user capability checks.
Additional articles and tutorials that we recommend checking out include our guide on speeding up WordPress, solutions to forgotten WordPress page speed problems, WordPress troubleshooting guide, and Mario Peshev’s article on scaling mistakes when running a large WordPress site.
Large WordPress Site Performance Issue #3 – Site Content
Large WordPress sites typically have loads of content. In Charly’s case, for example, the client’s website had over 900k entries.
If you’ve gone and optimized the database and you’re still experiencing issues, here are some of the things you can look at:
- Perform a content audit: A content audit is essentially performing an inventory of your existing content and assessing and identifying content that’s outdated, obsolete, duplicated, etc, before deciding what to do with it (e.g. update, SEO optimize, trash). It’s a long-term but effective and important strategy for keeping your site’s content manageable and maintained.
- Use lazy loading: Lazy loading can help to ensure that media files are only loaded when they are needed, which can significantly improve page load times.
- Use a content delivery network (CDN): Consider using a content delivery network (CDN) to distribute cached media files and reduce the load on your server. A CDN can help speed up your website by caching your website’s content on servers located around the world, reducing the load on your server. Popular CDNs include Cloudflare and MaxCDN. Note that all WPMU DEV membership and hosting plans include a CDN. Our Hummingbird and Smush plugins also include a CDN (Hummingbird also offers Cloudflare integration).
- Use content optimization plugins: Optimize images, videos, and other media files by compressing them and reducing their file size. If the site contains loads of images, consider using an image optimization plugin like Smush, which significantly reduces image file sizes without compromising on image quality to improve content delivery performance. Smush also includes WPMU DEV’s CDN.
- Use a managed WordPress hosting service: A managed WordPress hosting service can provide you with optimized servers and database management tools to help keep your website running smoothly. As discussed in the next section below, WPMU DEV not only offers a best-of-class managed WordPress hosting service, but it is also specifically configured to deliver enterprise-level hosting for WordPress sites of all kinds and sizes.
Large WordPress Site Performance Issue #4 – Hosting
If you are still experiencing problems with the site after fixing issues with the WordPress database and optimizing the site’s core, plugins, themes, and content, the issue may be related to web hosting.
Consider using a managed WordPress hosting service with a company that specializes in WordPress.
Hosting with a reputable host not only means placing your site in the care of an experienced team who will handle areas like server optimization and database management for you, but also migrate your existing website to their servers.
This is very important, as a large WordPress site no doubt has lots of moving parts and active traffic and transactional events taking place, and you don’t want to lose any valuable data or break anything during the migration process.
Additional hosting considerations for a large WordPress site include the ability to handle demands with ample resources, uptime, speed, and customer support.
WPMU DEV offers enterprise-level hosting, 24/7 expert hosting and WordPress support, and migrations by a team of experts who will handle everything for you, including troubleshooting any potential issues with your site.
Additionally, WPMU DEV has been independently rated and reviewed by many users as one of the leading managed WordPress hosting companies, with a near-perfect rating score. G2.com, for example, rates WPMU DEV 4.8 out of 5 stars overall, and 9.8 out of 10 for quality of support.
More importantly and on a practical level, our expert team proactively manages larger sites by regularly checking areas like “PHP error logs” for any errors in the plugins, themes or in the WordPress core and “PHP slow logs” for slow loading scripts (e.g. plugins where scripts exceed 30 seconds to execute), access logs (to see if there’s a DDoS attack or high visitors in general), and load on the server resources, including CPU, RAM, etc.
The team also checks if WAF is enabled, caching is ON, and any non-used profiling software is turned off when not needed, and will perform conflict tests for plugins and themes and run query monitoring scans at the mysql level when required.
We also offer integration with New Relic and Blackfire to profile the site and its pages for all sites, large and small.
Managing Larger WordPress Sites Is A Big Job
A large WordPress site differs from other WordPress sites mostly in the scale and complexity of its management.
Dealing with performance issues in large, complex WordPress sites requires having the skills and the expertise to handle challenges and meet the high expectations of enterprise clients.
Finding information on WordPress enterprise development can be challenging, but focusing on different areas like studying the WordPress Core, APIs, and the surrounding ecosystem, practicing in the main technical areas, and following leading agencies, will help you become more knowledgeable and confident in your abilities as a developer.
Also, managing a large WordPress database can be challenging but there are solutions available to help you manage it. By optimizing your database, using caching and CDN services, using a managed WordPress hosting service, and regularly backing up your database, you can ensure that your website runs smoothly and avoid potential issues.
By addressing common performance challenges and regularly monitoring your site’s performance to identify and address any issues as they arise, you can significantly improve the performance of your larger WordPress site.
Finally, hosting your site on enterprise-level servers with an experienced and reliable managed WordPress hosting partner like WPMU DEV will not only improve your large site’s performance but also help to eliminate problems and issues, as your site will be expertly managed and monitored 24/7.
If you are looking to migrate your existing site from another host or upgrade hosting for a large WordPress site, we recommend looking at our enterprise-level hosting plans (3 x Essential and 3 x Premium options), and taking advantage of our hosting buyout and free expert site migration service.
***
Special thanks to WPMU DEV member Charly Leethan for her contribution to this post. AskCharlyLeethan provides ongoing support and advice to help small businesses define and refine their processes and plan and build their web presence using current and emerging technologies.
