• Home
  • -
  • PHP8: Benchmarking the Symfony Demo App with and without JIT

PHP8: Benchmarking the Symfony Demo App with and without JIT

I was recently asked whether JIT will bring a big improvement to our PHP applications and I decided to put that to the test. So here's a short test of PHP 8 performance with different configurations.

The Test Setup

Symfony Demo App

Symfony with Doctrine is pretty much our go-to setup for most applications, so the demo application should give me a pretty good idea of what impact PHP 8 will have on our applications.

The Results

PHP 8 performance benchmark without OPCache

Let's first establish a baseline. PHP8 with out-of-the-box configuration with opcache disabled:

$ ab -n 100 -c 20 -l https://localhost:8000/en/blog/
This is ApacheBench, Version 2.3 <$Revision: 1843412 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Document Path:          /en/blog/
Document Length:        Variable

Concurrency Level:      20
Time taken for tests:   18.510 seconds
Complete requests:      100
Failed requests:        0
Total transferred:      8194800 bytes
HTML transferred:       8153300 bytes
Requests per second:    5.40 [#/sec] (mean)
Time per request:       3701.971 [ms] (mean)
Time per request:       185.099 [ms] (mean, across all concurrent requests)
Transfer rate:          432.35 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        3    5   4.4      3      19
Processing:   193 3308 915.8   3652    3826
Waiting:      191 3308 915.9   3651    3825
Total:        197 3313 913.3   3654    3829

Percentage of the requests served within a certain time (ms)
  50%   3654
  66%   3736
  75%   3771
  80%   3778
  90%   3802
  95%   3816
  98%   3828
  99%   3829
 100%   3829 (longest request)

Seems pretty slow to me - 5.4 requests per second

PHP 8 performance benchmark with OPCache

Let's set the following configuration options to enable OPCache and restart the symfony app:

zend_extension=opcache.so
opcache.enable=1
opcache.enable_cli=1

Let's test again:

$ ab -n 100 -c 20 -l https://localhost:8000/en/blog/
This is ApacheBench, Version 2.3 <$Revision: 1843412 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Document Path:          /en/blog/
Document Length:        Variable

Concurrency Level:      20
Time taken for tests:   3.674 seconds
Complete requests:      100
Failed requests:        0
Total transferred:      8194800 bytes
HTML transferred:       8153300 bytes
Requests per second:    27.22 [#/sec] (mean)
Time per request:       734.707 [ms] (mean)
Time per request:       36.735 [ms] (mean, across all concurrent requests)
Transfer rate:          2178.48 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        2    5   3.7      3      14
Processing:    51  609 151.3    668     697
Waiting:       51  609 151.3    668     697
Total:         58  614 149.0    671     711

Percentage of the requests served within a certain time (ms)
  50%    671
  66%    675
  75%    681
  80%    685
  90%    688
  95%    689
  98%    691
  99%    711
 100%    711 (longest request)

Ok, that's a lot faster - we are now at 27.22 requests per second

PHP 8 performance benchmark with OPCache and JIT

To answer the original question on whether or not JIT will help our applications, I then added JIT with the following settings in php.ini:

opcache.jit_buffer_size=100M
opcache.jit=1235

(You can look up the config reference and defaults at https://wiki.php.net/rfc/jit#phpini_defaults )

Now, let's benchmark again:

$ ab -n 100 -c 20 -l https://localhost:8000/en/blog/
This is ApacheBench, Version 2.3 <$Revision: 1843412 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Document Path:          /en/blog/
Document Length:        Variable

Concurrency Level:      20
Time taken for tests:   3.756 seconds
Complete requests:      100
Failed requests:        0
Total transferred:      8194800 bytes
HTML transferred:       8153300 bytes
Requests per second:    26.63 [#/sec] (mean)
Time per request:       751.111 [ms] (mean)
Time per request:       37.556 [ms] (mean, across all concurrent requests)
Transfer rate:          2130.91 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        2    4   3.6      3      15
Processing:    64  612 148.7    673     692
Waiting:       63  612 148.7    673     692
Total:         71  616 146.2    675     706

Percentage of the requests served within a certain time (ms)
  50%    675
  66%    680
  75%    682
  80%    683
  90%    686
  95%    687
  98%    687
  99%    706
 100%    706 (longest request)

Huh, that's actually a bit slower than without JIT configuration. But to be honest, I wasn't expecting much of a difference (see below). We are now at 26.63 requests per second, which seems like more or less the same result as before.

Comparison, Result, Conclusion

Configuration Requests
PHP8 without OPCache 5.4 rq/s
PHP8 with OPCache 27.2 rq/s
PHP8 with OPCache + JIT 26.6 rq/s

The point is, if you look in the announcements and comments about what the JIT does you can pretty much guess why this is kind of the expected result. The JIT compiler improves performance for CPU-intensive work (for example calculating Mandelbrots) but not so much for the "default" kind of applications. We usually don't do all that many calculations.

Upshot: We won't see an immediate improvement of our default use cases, but there are two things I'd like to point out before you go dismissing the JIT out of hand:

  • it has just been introduced and will most likely be improved further in the future, so it might improve our setups after all
  • the CPU computation performance improvements mean that PHP might now be able to compete in a field it did not compete in before - we can now probably use PHP in areas where we would have had to switch to a different language for decent performance before now

All in all, enabling JIT does not seem to harm our applications and might even help in high computational load scenarios.

Sidenote: Comparison with PHP 7.4
I also did the same benchmark on the same Symfony app with PHP 7.4: there is no notable difference in the performance between the two versions.

Additional Infos