+1 (276) 265-7197
StackPioneer

From One LAMP Box to Two App Servers: The Four Things That Break

Sooner or later a single LAMP server stops being enough — not always for capacity. More often it is that one reboot, one failed disk, or one kernel update takes the whole business offline, and someone finally asks what it would take to have two.

The load balancer is the easy part. An hour with an ALB, HAProxy, or a second Apache doing mod_proxy_balancer and traffic is split. What breaks is everything in the application that quietly assumed there was exactly one machine. In a decade-old PHP app that is usually four things: sessions, uploads, cron, and caches. Find them before you split traffic, not after.

First, decide whether you need this

Honest answer first: if your problem is that the app is slow, two servers will not fix it. Two copies of an application that spends 900 ms in an unindexed query are two copies of the same 900 ms, and you have doubled your deploy surface and your config drift. Tune first; see the posts on query tuning, caching, and PHP-FPM sizing.

Scale out when one of these is true:

  • Availability. You need to patch, reboot, or lose a node without downtime.
  • Real CPU saturation. sar -u 1 60 shows sustained high %user across cores during normal traffic, PHP-FPM's listen queue is non-zero, and you have already fixed the slow requests.
  • Deployment risk. You want to drain one node, deploy, verify, then move the other.

If it is availability you want, note that two app nodes with one MySQL box still have a single point of failure. That is fine as a first step — most outages are app-node outages — but say it out loud so nobody thinks they bought HA.

1. Sessions on local disk

Default PHP writes sessions to files under /var/lib/php/sessions. Node A writes the session; the next request lands on node B; the user is logged out. Sticky sessions hide this until a node is drained, and then every logged-in user is dropped at once.

Check what you have:

php -i | grep -E 'session.save_(handler|path)'
grep -rn 'session_set_save_handler\|session.save_path' /var/www --include=*.php --include=*.ini

Move them to Redis. In /etc/php/8.3/fpm/php.ini (or a pool-level override):

session.save_handler = redis
session.save_path = "tcp://10.0.1.20:6379?auth=...&database=0"
session.gc_maxlifetime = 86400

Then set maxmemory-policy volatile-lru on that Redis instance — never allkeys-lru on a database holding sessions, or you will evict logins under memory pressure. Give sessions their own Redis database (or their own instance) so a cache flush does not log out every customer.

If the app rolled its own session handler into MySQL, leave it there for now. It works across nodes already, and swapping it is a separate, optional change.

2. Uploads written into the document root

The second-most common assumption: move_uploaded_file() into /var/www/app/public/uploads. Node A has the file, node B returns 404, and your CDN caches the 404.

Find the writes:

grep -rn 'move_uploaded_file\|file_put_contents\|imagejpeg\|fopen(.*w' /var/www/app --include=*.php | grep -v vendor/

Three workable options, in increasing order of effort:

  1. Shared filesystem (EFS, or NFS from a storage node). Cheapest change: the path stays the same, the app does not. Watch latency — a per-request file_exists() on NFS is noticeably slower than local disk.
  2. Object storage behind a small abstraction. Write a Storage::put()/Storage::url() pair, change the handful of call sites you just grepped, and serve from S3 or a CDN. This is the right long-term answer and is usually a day or two of work, not a rewrite.
  3. One-way rsync from a designated writer node. Acceptable as a stopgap for assets that are written rarely; do not use it for anything a user expects to see immediately after upload.

Whichever you pick, migrate the existing files first and verify counts and checksums before cutting traffic over.

3. Cron running twice

Copy the server image, and you copy the crontab. Now the nightly invoice job runs on both nodes at 02:00, and either you double-charge somebody or you deadlock on the same rows.

List what exists on each node:

crontab -l; ls -l /etc/cron.d/; systemctl list-timers --all

Pick one of these and be explicit:

  • A designated cron node. Simplest and usually correct: cron runs only on node A, which is also configured to stay in the pool. Document it, and put a check in your monitoring that the job actually ran — a silent cron node is a classic 3-a.m. surprise.

  • A lock the job takes itself. flock on a shared filesystem path, or an advisory lock in MySQL:

    SELECT GET_LOCK('invoices:nightly', 0);

    Returns 1 to the winner, 0 to the loser, which exits. Release in a finally. This survives someone cloning the node again, which the designated-node approach does not.

While you are in there, check for jobs that assume local temp files from an earlier step in the same pipeline.

4. Caches that only clear on one node

APCu, OPcache, and file-based application caches are per-process and per-node. A deploy or an admin "clear cache" button clears node A and leaves node B serving stale config, stale routes, or stale prices.

Fixes, in order of safety:

  • Move shared application cache (config, query results, rendered fragments) to Redis or Memcached so there is one copy.
  • Keep OPcache local — it should be local — and make deploys invalidate it per node. Atomic symlink deploys with opcache_reset() fired on each node, or opcache.validate_timestamps=1 with a short revalidate_freq during the transition.
  • Delete any in-app "clear cache" button that only hits the local node, or wire it to the shared store. A button that lies is worse than no button.

The parts that are genuinely boring

MySQL. Both nodes now connect over the network, so raise max_connections deliberately (pm.max_children × nodes, plus headroom for cron and your own CLI sessions), bind MySQL to the private subnet only, and use per-node application users so you can see in SHOW PROCESSLIST who is doing what.

Config drift. Two hand-configured servers become two different servers within a month. Put /etc/php, /etc/apache2, and your deploy scripts under Ansible — or at minimum, build node B from a documented script and diff php -i and apachectl -S output between the nodes before go-live.

Health checks. Point the balancer at a route that touches MySQL and Redis and returns 200 only if both answer, not at / and not at a static file. A node whose database connection is dead should leave the pool on its own.

Logs. tail -f on one box no longer tells you anything. Ship Apache and PHP error logs somewhere central, and add the node name to the access log format so you can tell which server served a bad request.

A cutover that you can undo

  1. Build node B from a script; diff its php -i, module list, and apachectl -S against node A.
  2. Fix sessions, uploads, cron, and caches while still serving all traffic from node A. Each change is independently deployable and independently reversible.
  3. Add node B to the balancer at 5% weight. Watch error rates, session loss, and upload 404s for a day.
  4. Go to 50/50. Drain node A for an hour as a rehearsal — this is the test that matters, because draining is what you bought.
  5. Write down which node runs cron and how to fail it over.

If steps 2 and 3 are quiet, the rest is uneventful. If they are not, you have found the assumption that would have taken the site down at 50/50, on a day you were watching.

Most of this work is grep, configuration, and a handful of call-site changes — days, not a rewrite. If you want a second pair of eyes on which of the four your application actually has, tell us what you are running and we will tell you what it would take.