Drupal 8 Performance: Moving the service container cache away from the database

Drupal relies on pluggable cache backends to store cache data such as Memcache, Wincache, Database, etc. The default storage backend is the Database, but Drupal being a very cache intensive application (even more in Drupal 8) you want to get better performance by using faster backends that will yield lower latency and scale better.

Moving caching away from the database is done by replacing the caching services by ones that do not rely on the database. You define the services in your services.yml file and the binaries routing in settings.php:

Services.yml

[...]

services:
  cache.backend.wincache:
    class: Drupal\wincache\WincacheBackendFactory 
    arguments: ['@app.root', '@site.path', '@cache_tags.invalidator.checksum']
  cache_tags.invalidator.checksum:
    class: Drupal\wincache\WincacheTagChecksum
    tags:
      - { name: cache_tags_invalidator}

Settings.php

$settings['cache']['bins']['bootstrap'] = 'cache.backend.wincache';
$settings['cache']['bins']['render'] = 'cache.backend.wincache';
$settings['cache']['bins']['discovery'] = 'cache.backend.wincache';
$settings['cache']['bins']['data'] = 'cache.backend.wincache';
$settings['cache']['bins']['config'] = 'cache.backend.wincache';
$settings['cache']['bins']['container'] = 'cache.backend.wincache';
$settings['cache']['bins']['toolbar'] = 'cache.backend.wincache';
$settings['cache']['bins']['menu'] = 'cache.backend.wincache';
$settings['cache']['bins']['entity'] = 'cache.backend.wincache';
$settings['cache']['bins']['default'] = 'cache.backend.wincache';

With the above code we are moving a big bunch of caching backends to in-memory. One thing I noticed is that even after routing “everything” to non database storage the service container was still being loaded from the database.

This happens because a streamlined service container (the bootstrap service container) is used before loading the service container itself, and this container has no knowledge about our services.

The bootstrap container is hardcoded into Drupal Core but can be overriden in settings.php:

if (class_exists(\Composer\Autoload\ClassLoader::class)) {
  $loader = new \Composer\Autoload\ClassLoader();
  $loader->addPsr4('Drupal\\wincachedrupal\\', 'modules/wincachedrupal/src');
  $loader->register();
  
  $settings['bootstrap_container_definition'] = [
    'parameters' => [],
    'services' => [
      'cache.container' => [
        'class' => 'Drupal\wincachedrupal\Cache\WincacheBackend',
        'arguments' => ['container', $settings['hash_salt'] ,'@cache_tags_provider.container'],
      ],
      'cache_tags_provider.container' => [
        'class' => 'Drupal\wincachedrupal\Cache\WincacheTagChecksum',
      ],
    ],
  ];
}

Notice that we had to manually register the namespace for the clases that are being used in the bootstrap container because Drupal’s autoloading service is still not available in such an early bootstrap phase.

We could also modify the composer.json file so that Wincache’s namespace was included in composer’s autoload, but that just seems to fragile.

Another posiblity is to move the service container to Couchbase:

$settings['bootstrap_container_definition'] = [
    'parameters' => [],
    'services' => [
      'settings' => [
        'class' => 'Drupal\Core\Site\Settings',
        'factory' => 'Drupal\Core\Site\Settings::getInstance',
      ],
      'couchbase.manager' => [
        'class' => 'Drupal\couchbasedrupal\CouchbaseManager',
        'arguments' => ['@settings'],
      ],
      'cache.rawbackend.couchbase' => [
        'class' => 'Drupal\couchbasedrupal\Cache\CouchbaseRawBackendFactory',
        'arguments' => ['@couchbase.manager', $settings['hash_salt'],  $settings['hash_salt']],
      ],
      'cache.container' => [
        'class' => 'Drupal\supercache\Cache\CacheRawBackendInterface',
        'factory' => ['@cache.rawbackend.couchbase', 'get'],
        'arguments' => ['container'],
      ],
    ],
  ];

if (class_exists(\Composer\Autoload\ClassLoader::class)) {
  $loader = new \Composer\Autoload\ClassLoader();
  $loader->addPsr4('Drupal\\wincachedrupal\\', 'modules/wincachedrupal/src');
  $loader->addPsr4('Drupal\\couchbasedrupal\\', 'modules/couchbasedrupal/src');
  $loader->addPsr4('Drupal\\supercache\\', 'modules/supercache/src');
  $loader->register();
}

// Couchbase server.
$settings['couchbase_settings'] =
  array('servers' =>
    array('default' =>
      array(
        'uri' => 'couchbase://127.0.0.1'
         )
	)
  );

FROM QUESTIONS IN THE COMMENTS

I did not measure the exact performance improvement of this change. But I did evaluate the overal performance improvent of moving everything – including the TagChecksum which we had to reimplement – off the database. The biggest performance gain came from moving TagChecksum storage off the database. The problem is that you can’t store your tag checksum in a volatile storage, so this was quite an experimental performance test. Hiting the database as little as possible and using proximity in-memory storage is the key to get an application properly performing on cloud setups.

Seriusly, I don’t think this can be considered a bug. Understand what you are deploying and how it works, and be happy that you have the flexibility to change and override things.

 

click here 3501
click here 3502
click here 3503
click here 3504
click here 3505
click here 3506
click here 3507
click here 3508
click here 3509
click here 3510
click here 3511
click here 3512
click here 3513
click here 3514
click here 3515
click here 3516
click here 3517
click here 3518
click here 3519
click here 3520
click here 3521
click here 3522
click here 3523
click here 3524
click here 3525
click here 3526
click here 3527
click here 3528
click here 3529
click here 3530
click here 3531
click here 3532
click here 3533
click here 3534
click here 3535
click here 3536
click here 3537
click here 3538
click here 3539
click here 3540
click here 3541
click here 3542
click here 3543
click here 3544
click here 3545
click here 3546
click here 3547
click here 3548
click here 3549
click here 3550
click here 3551
click here 3552
click here 3553
click here 3554
click here 3555
click here 3556
click here 3557
click here 3558
click here 3559
click here 3560
click here 3561
click here 3562
click here 3563
click here 3564
click here 3565
click here 3566
click here 3567
click here 3568
click here 3569
click here 3570
click here 3571
click here 3572
click here 3573
click here 3574
click here 3575
click here 3576
click here 3577
click here 3578
click here 3579
click here 3580
click here 3581
click here 3582
click here 3583
click here 3584
click here 3585
click here 3586
click here 3587
click here 3588
click here 3589
click here 3590
click here 3591
click here 3592
click here 3593
click here 3594
click here 3595
click here 3596
click here 3597
click here 3598
click here 3599
click here 3600
click here 3601
click here 3602
click here 3603
click here 3604
click here 3605
click here 3606
click here 3607
click here 3608
click here 3609
click here 3610
click here 3611
click here 3612
click here 3613
click here 3614
click here 3615
click here 3616
click here 3617
click here 3618
click here 3619
click here 3620
click here 3621
click here 3622
click here 3623
click here 3624
click here 3625
click here 3626
click here 3627
click here 3628
click here 3629
click here 3630
click here 3631
click here 3632
click here 3633
click here 3634
click here 3635
click here 3636
click here 3637
click here 3638
click here 3639
click here 3640
click here 3641
click here 3642
click here 3643
click here 3644
click here 3645
click here 3646
click here 3647
click here 3648
click here 3649
click here 3650
click here 3651
click here 3652
click here 3653
click here 3654
click here 3655
click here 3656
click here 3657
click here 3658
click here 3659
click here 3660
click here 3661
click here 3662
click here 3663
click here 3664
click here 3665
click here 3666
click here 3667
click here 3668
click here 3669
click here 3670
click here 3671
click here 3672
click here 3673
click here 3674
click here 3675
click here 3676
click here 3677
click here 3678
click here 3679
click here 3680
click here 3681
click here 3682
click here 3683
click here 3684
click here 3685
click here 3686
click here 3687
click here 3688
click here 3689
click here 3690
click here 3691
click here 3692
click here 3693
click here 3694
click here 3695
click here 3696
click here 3697
click here 3698
click here 3699
click here 3700
click here 3701
click here 3702
click here 3703
click here 3704
click here 3705
click here 3706
click here 3707
click here 3708
click here 3709
click here 3710
click here 3711
click here 3712
click here 3713
click here 3714
click here 3715
click here 3716
click here 3717
click here 3718
click here 3719
click here 3720
click here 3721
click here 3722
click here 3723
click here 3724
click here 3725
click here 3726
click here 3727
click here 3728
click here 3729
click here 3730
click here 3731
click here 3732
click here 3733
click here 3734
click here 3735
click here 3736
click here 3737
click here 3738
click here 3739
click here 3740
click here 3741
click here 3742
click here 3743
click here 3744
click here 3745
click here 3746
click here 3747
click here 3748
click here 3749
click here 3750
click here 3751
click here 3752
click here 3753
click here 3754
click here 3755
click here 3756
click here 3757
click here 3758
click here 3759
click here 3760
click here 3761
click here 3762
click here 3763
click here 3764
click here 3765
click here 3766
click here 3767
click here 3768
click here 3769
click here 3770
click here 3771
click here 3772
click here 3773
click here 3774
click here 3775
click here 3776
click here 3777
click here 3778
click here 3779
click here 3780
click here 3781
click here 3782
click here 3783
click here 3784
click here 3785
click here 3786
click here 3787
click here 3788
click here 3789
click here 3790
click here 3791
click here 3792
click here 3793
click here 3794
click here 3795
click here 3796
click here 3797
click here 3798
click here 3799
click here 3800
click here 3801
click here 3802
click here 3803
click here 3804
click here 3805
click here 3806
click here 3807
click here 3808
click here 3809
click here 3810
click here 3811
click here 3812
click here 3813
click here 3814
click here 3815
click here 3816
click here 3817
click here 3818
click here 3819
click here 3820
click here 3821
click here 3822
click here 3823
click here 3824
click here 3825
click here 3826
click here 3827
click here 3828
click here 3829
click here 3830
click here 3831
click here 3832
click here 3833
click here 3834
click here 3835
click here 3836
click here 3837
click here 3838
click here 3839
click here 3840
click here 3841
click here 3842
click here 3843
click here 3844
click here 3845
click here 3846
click here 3847
click here 3848
click here 3849
click here 3850
click here 3851
click here 3852
click here 3853
click here 3854
click here 3855
click here 3856
click here 3857
click here 3858
click here 3859
click here 3860
click here 3861
click here 3862
click here 3863
click here 3864
click here 3865
click here 3866
click here 3867
click here 3868
click here 3869
click here 3870
click here 3871
click here 3872
click here 3873
click here 3874
click here 3875
click here 3876
click here 3877
click here 3878
click here 3879
click here 3880
click here 3881
click here 3882
click here 3883
click here 3884
click here 3885
click here 3886
click here 3887
click here 3888
click here 3889
click here 3890
click here 3891
click here 3892
click here 3893
click here 3894
click here 3895
click here 3896
click here 3897
click here 3898
click here 3899
click here 3900
click here 3901
click here 3902
click here 3903
click here 3904
click here 3905
click here 3906
click here 3907
click here 3908
click here 3909
click here 3910
click here 3911
click here 3912
click here 3913
click here 3914
click here 3915
click here 3916
click here 3917
click here 3918
click here 3919
click here 3920
click here 3921
click here 3922
click here 3923
click here 3924
click here 3925
click here 3926
click here 3927
click here 3928
click here 3929
click here 3930
click here 3931
click here 3932
click here 3933
click here 3934
click here 3935
click here 3936
click here 3937
click here 3938
click here 3939
click here 3940
click here 3941
click here 3942
click here 3943
click here 3944
click here 3945
click here 3946
click here 3947
click here 3948
click here 3949
click here 3950
click here 3951
click here 3952
click here 3953
click here 3954
click here 3955
click here 3956
click here 3957
click here 3958
click here 3959
click here 3960
click here 3961
click here 3962
click here 3963
click here 3964
click here 3965
click here 3966
click here 3967
click here 3968
click here 3969
click here 3970
click here 3971
click here 3972
click here 3973
click here 3974
click here 3975
click here 3976
click here 3977
click here 3978
click here 3979
click here 3980
click here 3981
click here 3982
click here 3983
click here 3984
click here 3985
click here 3986
click here 3987
click here 3988
click here 3989
click here 3990
click here 3991
click here 3992
click here 3993
click here 3994
click here 3995
click here 3996
click here 3997
click here 3998
click here 3999
click here 4000